[2024-09-11 06:20:31.745590] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69703144) [2024-09-11 06:20:31.746484] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4977, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:31.746691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035631746197000, start_id=1726035631746197000, end_id=1726035631746197001) [2024-09-11 06:20:31.747100] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:31.747111] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:31.747120] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:31.747128] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:31.747867] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.747898] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.747914] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:31.747928] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:31.747934] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:31.747941] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.747953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.747965] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:31.747968] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.747974] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.747977] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:31.747982] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.747994] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.748005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:31.748008] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.748010] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.748012] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:31.748039] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035631748016), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:31.748048] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:31.748056] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:31.748063] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2476, ddl_arg=NULL) [2024-09-11 06:20:31.850132] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69598603) [2024-09-11 06:20:31.850931] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4978, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:31.851097] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035631850575000, start_id=1726035631850575000, end_id=1726035631850575001) [2024-09-11 06:20:31.851489] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:31.852161] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852191] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852206] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:31.852219] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:31.852226] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:31.852233] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852244] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852255] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:31.852258] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.852263] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.852266] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:31.852269] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852281] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.852293] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:31.852295] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.852298] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.852300] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:31.852321] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035631852304), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:31.852332] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:31.852343] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:20:31.954651] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69494073) [2024-09-11 06:20:31.955441] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4979, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:31.955592] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035631954715000, start_id=1726035631954715000, end_id=1726035631954715001) [2024-09-11 06:20:31.956028] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:31.956042] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:31.956051] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:31.956754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956810] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:31.956820] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:31.956826] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:31.956831] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956855] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956867] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:31.956871] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.956875] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.956878] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:31.956883] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956895] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:31.956903] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:31.956906] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:31.956908] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:31.956911] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:31.956942] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035631956915), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:31.956954] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:31.956964] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.059705] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69389020) [2024-09-11 06:20:32.060547] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4980, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.060715] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035632060074000, start_id=1726035632060074000, end_id=1726035632060074001) [2024-09-11 06:20:32.061103] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.061116] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.061122] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.061818] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061851] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061867] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.061878] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.061884] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.061893] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061917] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.061920] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.061926] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.061929] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.061934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061944] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.061952] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.061955] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.061960] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.061962] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.061986] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632061966), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.061994] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.062001] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.062007] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2305, ddl_arg=NULL) [2024-09-11 06:20:32.164378] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69284348) [2024-09-11 06:20:32.165144] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4981, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.165332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1726035632165214000, start_id=1726035632165214000, end_id=1726035632165214001) [2024-09-11 06:20:32.165388] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=44, avg_log_batch_cnt=1, total_group_log_size=36805, avg_group_log_size=836) [2024-09-11 06:20:32.165400] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=36966) [2024-09-11 06:20:32.165752] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.165765] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.165799] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.166453] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166482] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166498] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.166507] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.166515] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.166520] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166543] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.166546] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.166552] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.166556] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.166560] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166571] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.166580] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.166584] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.166589] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.166592] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.166613] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632166596), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.166622] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.166630] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.166634] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:20:32.269117] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69179624) [2024-09-11 06:20:32.269960] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4982, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.270138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035632269456000, start_id=1726035632269456000, end_id=1726035632269456001) [2024-09-11 06:20:32.270581] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.270595] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.270603] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.271261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271291] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271306] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.271318] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.271324] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.271331] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271343] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271360] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.271364] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.271370] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.271374] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.271377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271387] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.271396] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.271398] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.271401] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.271404] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.271426] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632271407), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.271436] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.271441] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.271445] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2330, ddl_arg=NULL) [2024-09-11 06:20:32.373870] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=69074888) [2024-09-11 06:20:32.374663] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4983, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.374852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035632374485000, start_id=1726035632374485000, end_id=1726035632374485001) [2024-09-11 06:20:32.375307] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.375321] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.375329] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.376034] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.376104] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.376110] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.376117] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376129] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376145] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.376147] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.376153] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.376163] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.376166] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376177] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.376188] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.376190] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.376193] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.376195] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.376223] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632376201), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.376231] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.376238] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.376245] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:32.478626] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68970100) [2024-09-11 06:20:32.479386] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4984, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.479547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035632478657000, start_id=1726035632478657000, end_id=1726035632478657001) [2024-09-11 06:20:32.479952] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.479965] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.479974] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.480607] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480652] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.480663] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.480669] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.480674] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480699] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.480702] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.480708] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.480712] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.480721] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480737] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.480750] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.480753] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.480756] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.480759] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.480803] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632480764), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.480812] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.480818] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.480823] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2199, ddl_arg=NULL) [2024-09-11 06:20:32.593939] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68854811) [2024-09-11 06:20:32.594720] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4985, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.594927] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035632594372000, start_id=1726035632594372000, end_id=1726035632594372001) [2024-09-11 06:20:32.595396] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.595409] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.595416] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.596145] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.596180] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.596189] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.596213] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632596193), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.596221] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.596229] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.698694] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:32.699431] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4986, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.699599] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035632699525000, start_id=1726035632699525000, end_id=1726035632699525001) [2024-09-11 06:20:32.700020] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.700044] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.700052] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.700657] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.700684] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.700700] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.700713] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.700719] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.700740] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632700723), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.700748] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.700754] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.813768] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68634968) [2024-09-11 06:20:32.814667] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4987, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.814905] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035632813890000, start_id=1726035632813890000, end_id=1726035632813890001) [2024-09-11 06:20:32.815350] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:32.815361] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.815370] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:32.815377] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.816063] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816092] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:32.816117] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:32.816121] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:32.816128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816139] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:32.816154] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.816159] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.816165] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:32.816168] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816180] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.816190] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:32.816193] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:32.816195] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.816197] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:32.816223] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632816201), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.816234] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:32.816241] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:32.816246] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2481, ddl_arg=NULL) [2024-09-11 06:20:32.918456] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:32.919274] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4988, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:32.919447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035632919094000, start_id=1726035632919094000, end_id=1726035632919094001) [2024-09-11 06:20:32.919805] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:32.920467] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.920497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:32.920521] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.920529] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:32.920549] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035632920532), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:32.920557] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.023510] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68425218) [2024-09-11 06:20:33.024333] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4989, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.024527] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035633024463000, start_id=1726035633024463000, end_id=1726035633024463001) [2024-09-11 06:20:33.024946] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.024961] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.024969] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.025622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631064307, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025667] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.025678] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.025684] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.025690] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025701] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025713] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.025715] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.025722] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.025725] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.025731] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025740] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.025749] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.025752] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.025754] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.025756] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.025793] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633025762), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.025803] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.025808] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.025812] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2304, ddl_arg=NULL) [2024-09-11 06:20:33.130646] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68318073) [2024-09-11 06:20:33.131439] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4990, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.131612] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035633130619000, start_id=1726035633130619000, end_id=1726035633130619001) [2024-09-11 06:20:33.132042] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.132057] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.132068] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.132724] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.132804] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.132811] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.132816] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132828] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631206577, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132839] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.132842] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.132846] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.132850] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.132853] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.132875] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.132879] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.132881] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.132884] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.132911] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633132889), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.132919] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.132926] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.132933] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2288, ddl_arg=NULL) [2024-09-11 06:20:33.235317] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68213405) [2024-09-11 06:20:33.236131] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4991, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.236310] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035633235941000, start_id=1726035633235941000, end_id=1726035633235941001) [2024-09-11 06:20:33.236371] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=38, avg_log_batch_cnt=1, total_group_log_size=29538, avg_group_log_size=777) [2024-09-11 06:20:33.236382] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=29538) [2024-09-11 06:20:33.236718] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.236729] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.236738] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.237415] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237448] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237464] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.237476] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.237482] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.237489] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237500] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237517] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.237521] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.237525] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.237529] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.237532] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237544] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035631275302, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.237553] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.237555] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.237557] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.237560] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.237583] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633237564), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.237591] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.237601] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.237607] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2292, ddl_arg=NULL) [2024-09-11 06:20:33.340794] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68107942) [2024-09-11 06:20:33.341539] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4992, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.341715] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035633341101000, start_id=1726035633341101000, end_id=1726035633341101001) [2024-09-11 06:20:33.342160] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.342174] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.342182] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.342876] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.342905] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.342922] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.342933] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.342940] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.342945] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.342957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.342969] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.342972] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.342976] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.342979] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.342985] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.343008] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633342988), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.343018] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.343027] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.444148] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=68004588) [2024-09-11 06:20:33.445148] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4993, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.445371] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035633445258000, start_id=1726035633445258000, end_id=1726035633445258001) [2024-09-11 06:20:33.445827] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.445846] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.445856] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.446764] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=70] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446854] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.446866] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.446874] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.446883] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446902] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446921] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.446926] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.446931] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.446938] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.446946] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446965] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.446983] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.446987] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.446993] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.447000] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.447031] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633447008), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.447041] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.447052] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.447059] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2913, ddl_arg=NULL) [2024-09-11 06:20:33.548889] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67899845) [2024-09-11 06:20:33.549629] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4994, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.549838] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035633549656000, start_id=1726035633549656000, end_id=1726035633549656001) [2024-09-11 06:20:33.550297] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.550314] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.550326] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.550988] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551015] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551032] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.551042] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.551049] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.551054] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551077] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.551080] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.551084] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.551090] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.551093] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551105] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.551114] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.551119] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.551124] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.551127] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.551153] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633551135), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.551158] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.551164] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.551168] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:33.654873] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67793859) [2024-09-11 06:20:33.655764] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4995, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.656473] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.656487] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.656500] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.657211] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.657237] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.657255] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.657267] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.657274] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.657294] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633657277), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.657302] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.657310] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.759896] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67688832) [2024-09-11 06:20:33.760637] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4996, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.760807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035633759952000, start_id=1726035633759952000, end_id=1726035633759952001) [2024-09-11 06:20:33.761254] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.761269] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.761276] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.761916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.761943] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.761959] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.761970] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.761976] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.761989] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.762002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.762014] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.762017] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.762021] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.762026] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.762030] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.762042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.762052] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.762057] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.762059] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.762063] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.762085] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633762069), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.762091] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.762099] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.762103] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2209, ddl_arg=NULL) [2024-09-11 06:20:33.864936] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67583801) [2024-09-11 06:20:33.865676] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4997, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.865889] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035633865130000, start_id=1726035633865130000, end_id=1726035633865130001) [2024-09-11 06:20:33.866349] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:33.866364] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.866373] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.866381] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.867111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867144] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867159] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.867173] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.867180] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.867187] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867199] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867211] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.867214] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.867219] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.867223] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.867228] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867249] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.867261] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.867264] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.867266] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.867269] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.867295] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633867272), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.867304] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.867311] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.867317] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2382, ddl_arg=NULL) [2024-09-11 06:20:33.969872] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67478870) [2024-09-11 06:20:33.970627] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4998, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:33.970794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035633970445000, start_id=1726035633970445000, end_id=1726035633970445001) [2024-09-11 06:20:33.971224] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:33.971237] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:33.971246] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.971940] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.971972] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.971986] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:33.971996] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:33.972002] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:33.972007] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.972020] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.972032] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:33.972035] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.972040] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.972043] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:33.972046] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.972061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:33.972075] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:33.972077] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:33.972080] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:33.972082] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:33.972108] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035633972086), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:33.972116] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:33.972125] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:33.972129] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:20:34.074697] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67374032) [2024-09-11 06:20:34.075496] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=4999, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.075686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035634074709000, start_id=1726035634074709000, end_id=1726035634074709001) [2024-09-11 06:20:34.076108] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.076120] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.076126] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.076767] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076809] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076822] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.076833] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.076845] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.076850] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076863] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076873] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.076875] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.076879] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.076884] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.076888] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076898] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.076907] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.076909] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.076912] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.076914] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.076941] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634076918), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.076949] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.076957] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.076962] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2266, ddl_arg=NULL) [2024-09-11 06:20:34.179821] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67268913) [2024-09-11 06:20:34.180679] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5000, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.180891] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035634179841000, start_id=1726035634179841000, end_id=1726035634179841001) [2024-09-11 06:20:34.181346] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.181358] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.181367] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.182044] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182090] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.182103] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.182110] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.182116] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182140] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.182143] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.182147] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.182151] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.182156] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182168] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.182178] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.182182] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.182186] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.182192] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.182218] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634182195), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.182224] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.182231] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.182238] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2419, ddl_arg=NULL) [2024-09-11 06:20:34.284633] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67164103) [2024-09-11 06:20:34.285497] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5001, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.285718] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1726035634284944000, start_id=1726035634284944000, end_id=1726035634284944001) [2024-09-11 06:20:34.285803] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28255, avg_group_log_size=807) [2024-09-11 06:20:34.285814] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28255) [2024-09-11 06:20:34.286153] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.286165] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.286174] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.286880] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.286917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=36] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.286931] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.286941] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.286948] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.286953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.286966] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.286982] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.286985] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.286989] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.286993] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.286998] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.287010] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.287022] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.287025] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.287028] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.287030] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.287065] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634287043), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.287074] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.287081] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.287086] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2455, ddl_arg=NULL) [2024-09-11 06:20:34.389680] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=67059069) [2024-09-11 06:20:34.390603] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5002, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.390860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1726035634390127000, start_id=1726035634390127000, end_id=1726035634390127001) [2024-09-11 06:20:34.391315] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.391332] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.391341] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.392051] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392093] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.392103] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.392107] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.392113] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392123] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392133] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.392136] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.392140] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.392143] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.392146] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392154] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.392163] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.392165] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.392168] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.392170] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.392195] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634392175), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.392202] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.392207] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.392212] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2534, ddl_arg=NULL) [2024-09-11 06:20:34.495012] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66953721) [2024-09-11 06:20:34.495850] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5003, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.496038] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035634495250000, start_id=1726035634495250000, end_id=1726035634495250001) [2024-09-11 06:20:34.496481] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.496498] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.496504] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.497181] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497211] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497228] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.497241] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.497247] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.497252] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497265] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497275] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.497282] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.497285] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.497289] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.497292] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497302] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.497311] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.497313] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.497315] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.497318] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.497339] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634497321), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.497346] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.497355] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.497359] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2349, ddl_arg=NULL) [2024-09-11 06:20:34.601121] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66847614) [2024-09-11 06:20:34.602006] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5004, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.602185] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035634601590000, start_id=1726035634601590000, end_id=1726035634601590001) [2024-09-11 06:20:34.602666] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.602677] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.602685] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.603377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603406] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603419] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.603429] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.603435] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.603440] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603452] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603463] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.603466] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.603470] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.603473] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.603476] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603488] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.603496] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.603499] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.603502] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.603504] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.603524] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634603508), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.603532] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.603541] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.603545] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2426, ddl_arg=NULL) [2024-09-11 06:20:34.706329] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66742409) [2024-09-11 06:20:34.707137] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5005, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.707291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035634706835000, start_id=1726035634706835000, end_id=1726035634706835001) [2024-09-11 06:20:34.707715] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.707730] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.707738] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.708462] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708495] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708512] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.708524] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.708529] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.708536] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708547] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708558] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.708562] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.708567] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.708570] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.708573] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708582] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.708592] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.708595] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.708597] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.708600] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:34.708622] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634708604), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.708629] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.708636] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.811200] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66637534) [2024-09-11 06:20:34.812068] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5006, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.812283] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:34.812751] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.812765] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.812779] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.813424] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.813455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.813479] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.813487] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.813511] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634813491), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.813520] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.813527] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.813532] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2335, ddl_arg=NULL) [2024-09-11 06:20:34.916393] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66532337) [2024-09-11 06:20:34.917227] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5007, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:34.917392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035634917165000, start_id=1726035634917165000, end_id=1726035634917165001) [2024-09-11 06:20:34.917824] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:34.917836] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:34.917849] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:34.917857] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.918510] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918538] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918554] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:34.918565] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:34.918572] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:34.918577] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918588] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918601] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:34.918604] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.918607] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.918610] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:34.918614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918625] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:34.918636] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:34.918639] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:34.918643] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:34.918648] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:34.918670] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035634918652), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:34.918678] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:34.918684] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:34.918691] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2299, ddl_arg=NULL) [2024-09-11 06:20:35.021732] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66426996) [2024-09-11 06:20:35.022492] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5008, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.022659] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035635022453000, start_id=1726035635022453000, end_id=1726035635022453001) [2024-09-11 06:20:35.023116] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.023130] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:35.023808] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023837] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633064651, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023850] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.023860] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.023865] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.023869] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023881] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023893] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.023896] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.023900] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.023904] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.023911] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023927] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.023938] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.023941] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.023944] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.023947] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.023970] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635023952), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.023979] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.023987] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2257, ddl_arg=NULL) [2024-09-11 06:20:35.126966] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66321759) [2024-09-11 06:20:35.127761] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5009, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.127970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1726035635127835000, start_id=1726035635127835000, end_id=1726035635127835001) [2024-09-11 06:20:35.128397] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:35.129111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129142] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129155] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.129168] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.129174] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.129179] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129191] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633207036, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129203] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.129206] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.129209] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.129213] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.129218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129230] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.129242] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.129245] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.129247] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.129250] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.129272] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635129253), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.129281] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.129288] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2325, ddl_arg=NULL) [2024-09-11 06:20:35.231905] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66216819) [2024-09-11 06:20:35.232681] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5010, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.232846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035635232222000, start_id=1726035635232222000, end_id=1726035635232222001) [2024-09-11 06:20:35.233256] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.233268] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.233276] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.233928] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.233957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.233972] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.233982] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.233987] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.233995] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.234013] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.234024] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.234027] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.234031] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.234034] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.234036] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.234046] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035633275704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.234056] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.234059] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.234061] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.234063] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.234083] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635234066), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.234090] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.234097] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.234102] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2198, ddl_arg=NULL) [2024-09-11 06:20:35.336866] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66111867) [2024-09-11 06:20:35.337660] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5011, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.337846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035635337414000, start_id=1726035635337414000, end_id=1726035635337414001) [2024-09-11 06:20:35.338276] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.338292] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.338302] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.339060] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.339086] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:35.339111] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.339119] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.339141] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635339123), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.339151] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.339156] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.441849] INFO [RS] add_server (ob_root_service.cpp:7915) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=66006895) [2024-09-11 06:20:35.442618] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5012, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.442769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035635442640000, start_id=1726035635442640000, end_id=1726035635442640001) [2024-09-11 06:20:35.443131] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=42][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.443143] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.443152] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.443810] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443838] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443854] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.443865] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.443871] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.443877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443888] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443900] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.443903] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.443908] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.443912] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.443915] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443927] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.443938] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.443941] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.443944] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.443947] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.443971] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635443951), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.443979] INFO [RS] add_server (ob_root_service.cpp:7928) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.443985] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.443992] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2145, ddl_arg=NULL) [2024-09-11 06:20:35.546762] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65901966) [2024-09-11 06:20:35.547620] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5013, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.547837] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035635547676000, start_id=1726035635547676000, end_id=1726035635547676001) [2024-09-11 06:20:35.548270] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.548282] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.548292] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.548976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549006] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549022] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.549042] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.549048] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.549053] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549077] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.549080] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.549084] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.549088] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.549095] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549106] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.549117] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.549121] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.549124] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.549127] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.549153] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635549131), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.549162] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.549169] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.549174] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2415, ddl_arg=NULL) [2024-09-11 06:20:35.651794] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65796930) [2024-09-11 06:20:35.652585] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5014, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.652754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035635651943000, start_id=1726035635651943000, end_id=1726035635651943001) [2024-09-11 06:20:35.653222] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.653235] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.653243] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.653914] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.653943] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.653957] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.653971] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.653978] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.653983] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.653994] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.654006] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.654009] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.654013] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.654016] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.654022] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.654033] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.654050] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.654053] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.654056] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.654059] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.654082] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635654064), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.654090] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.654098] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.654103] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2310, ddl_arg=NULL) [2024-09-11 06:20:35.756783] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65691939) [2024-09-11 06:20:35.757561] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5015, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.757731] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035635757274000, start_id=1726035635757274000, end_id=1726035635757274001) [2024-09-11 06:20:35.758202] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.758215] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.758223] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.758895] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.758926] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.758942] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.758954] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.758960] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.758967] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.758978] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.758989] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.758992] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.758998] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.759001] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.759006] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.759017] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.759026] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.759029] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.759031] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.759034] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.759059] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635759040), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.759067] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.759075] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.759081] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2302, ddl_arg=NULL) [2024-09-11 06:20:35.862249] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65586481) [2024-09-11 06:20:35.863065] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5016, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.863274] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035635862380000, start_id=1726035635862380000, end_id=1726035635862380001) [2024-09-11 06:20:35.863691] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.863707] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.863716] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.864417] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864457] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864480] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.864491] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.864498] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.864505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864517] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864529] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.864533] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.864539] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.864543] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.864546] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864558] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.864570] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.864574] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.864578] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.864601] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635864582), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.864609] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.864618] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.864622] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2376, ddl_arg=NULL) [2024-09-11 06:20:35.971046] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65477679) [2024-09-11 06:20:35.971795] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5017, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:35.971948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035635971792000, start_id=1726035635971792000, end_id=1726035635971792001) [2024-09-11 06:20:35.972362] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:35.972372] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:35.972379] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:35.972385] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.972996] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973030] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973045] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:35.973057] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:35.973062] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:35.973069] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973090] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:35.973092] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.973095] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.973098] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:35.973102] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973113] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:35.973122] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:35.973124] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:35.973126] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:35.973128] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:35.973148] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035635973132), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:35.973156] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:35.973161] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:35.973166] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2122, ddl_arg=NULL) [2024-09-11 06:20:36.076960] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65371778) [2024-09-11 06:20:36.077707] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5018, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.077934] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035636077124000, start_id=1726035636077124000, end_id=1726035636077124001) [2024-09-11 06:20:36.078388] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.078401] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.078407] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.079098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079127] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079142] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.079153] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.079158] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.079163] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079177] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079186] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.079188] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.079192] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.079202] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.079205] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.079227] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:36.079230] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.079232] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.079234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:36.079256] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636079237), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.079264] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.079271] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.079279] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2321, ddl_arg=NULL) [2024-09-11 06:20:36.182411] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65266345) [2024-09-11 06:20:36.183209] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5019, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.183365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035636182488000, start_id=1726035636182488000, end_id=1726035636182488001) [2024-09-11 06:20:36.183822] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.183836] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.183842] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.184529] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184561] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184578] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.184590] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.184597] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.184602] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184623] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.184626] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.184629] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.184633] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.184637] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184653] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.184663] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:36.184666] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.184668] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.184670] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:36.184696] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636184675), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.184705] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.184714] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.184718] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2309, ddl_arg=NULL) [2024-09-11 06:20:36.287851] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65160908) [2024-09-11 06:20:36.288668] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5020, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.288913] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035636288625000, start_id=1726035636288625000, end_id=1726035636288625001) [2024-09-11 06:20:36.289443] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.289459] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.289471] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.290222] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290254] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290268] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.290280] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.290286] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.290293] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290308] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290324] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.290327] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.290330] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.290336] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.290339] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290349] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.290361] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:36.290363] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.290365] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.290368] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:36.290393] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636290371), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.290401] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.290409] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.290413] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2564, ddl_arg=NULL) [2024-09-11 06:20:36.403926] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=65044806) [2024-09-11 06:20:36.404765] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5021, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.404973] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1726035636404468000, start_id=1726035636404468000, end_id=1726035636404468001) [2024-09-11 06:20:36.405425] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.405440] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.405447] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.406128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.406163] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.406182] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.406193] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.406197] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.406201] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.406214] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.406225] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.406228] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.406232] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.406238] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.406241] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.406253] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.406279] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636406257), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.406289] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.406299] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.509123] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64939608) [2024-09-11 06:20:36.509933] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5022, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.510109] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035636509503000, start_id=1726035636509503000, end_id=1726035636509503001) [2024-09-11 06:20:36.510572] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.510587] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.510596] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.511247] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511279] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511295] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.511304] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.511310] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.511317] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511328] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511340] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.511342] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.511348] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.511353] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.511355] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.511376] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:36.511381] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.511384] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.511386] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:36.511409] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636511390), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.511417] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.511424] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.511431] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2310, ddl_arg=NULL) [2024-09-11 06:20:36.614090] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64834636) [2024-09-11 06:20:36.614878] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5023, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.615051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035636615011000, start_id=1726035636615011000, end_id=1726035636615011001) [2024-09-11 06:20:36.615479] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.615491] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.615499] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.616132] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616161] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616176] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.616187] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:36.616193] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:36.616201] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616212] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616224] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:36.616226] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.616229] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.616232] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:36.616235] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616246] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.616256] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:36.616259] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:36.616261] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.616264] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:36.616283] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636616267), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.616291] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.616296] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.616302] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2214, ddl_arg=NULL) [2024-09-11 06:20:36.718910] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64729818) [2024-09-11 06:20:36.719707] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5024, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.719902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035636719113000, start_id=1726035636719113000, end_id=1726035636719113001) [2024-09-11 06:20:36.720338] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.720350] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.720355] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.721023] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.721051] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.721067] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:36.721080] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.721088] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.721107] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636721091), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.721116] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.721121] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.823975] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:36.825369] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.825385] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.825396] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.826071] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.826082] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.826104] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636826086), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.826112] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.826121] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.928896] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64519832) [2024-09-11 06:20:36.929647] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5026, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:36.929854] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1726035636929658000, start_id=1726035636929658000, end_id=1726035636929658001) [2024-09-11 06:20:36.930275] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:36.930286] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:36.930294] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.930976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.931002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:36.931014] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:36.931028] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:36.931037] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.931041] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:36.931076] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035636931045), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:36.931085] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:36.931092] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:36.931117] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] [DDL] execute ddl like stmt(ret=-4122, cost=2224, ddl_arg=NULL) [2024-09-11 06:20:37.035280] INFO [RS] add_server (ob_root_service.cpp:7915) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64413446) [2024-09-11 06:20:37.036087] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5027, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.036264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035637035892000, start_id=1726035637035892000, end_id=1726035637035892001) [2024-09-11 06:20:37.036691] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:37.036702] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.036710] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.036719] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.037396] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037426] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635065005, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.703125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037442] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.037452] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.037458] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.037464] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037476] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037488] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.037491] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.037497] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.037501] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.037504] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037516] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.037532] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.037536] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.037539] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.037541] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.037576] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637037553), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.037585] INFO [RS] add_server (ob_root_service.cpp:7928) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.037594] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.037598] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55202][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2320, ddl_arg=NULL) [2024-09-11 06:20:37.140211] INFO [RS] add_server (ob_root_service.cpp:7915) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64308528) [2024-09-11 06:20:37.140967] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5028, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.141117] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035637140245000, start_id=1726035637140245000, end_id=1726035637140245001) [2024-09-11 06:20:37.141520] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.141532] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.141540] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.142194] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142223] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142236] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.142249] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.142254] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.142261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635207432, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142285] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.142287] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.142290] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.142295] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.142298] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035635276035, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.142318] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.142320] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.142322] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.142324] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.142344] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637142327), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.142350] INFO [RS] add_server (ob_root_service.cpp:7928) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.142357] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.142361] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55203][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2156, ddl_arg=NULL) [2024-09-11 06:20:37.245073] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=64203663) [2024-09-11 06:20:37.245903] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:37.246519] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.246533] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.246542] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.247195] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.247206] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.247230] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637247210), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.247237] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.247246] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.350328] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:37.351083] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5030, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.351245] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035637350809000, start_id=1726035637350809000, end_id=1726035637350809001) [2024-09-11 06:20:37.352427] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.352455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.352470] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.352482] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.352489] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.352494] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.352519] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637352500), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.352528] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.352539] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:20:37.454912] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63993816) [2024-09-11 06:20:37.455661] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5031, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.455831] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035637455137000, start_id=1726035637455137000, end_id=1726035637455137001) [2024-09-11 06:20:37.456230] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.456242] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.456250] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.456912] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.456941] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.456956] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.456966] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.456971] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.456978] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.456989] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.456998] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.457000] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.457005] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.457008] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.457011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.457020] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.457030] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.457033] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.457035] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.457038] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.457058] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637457043), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.457066] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.457073] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.457077] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2167, ddl_arg=NULL) [2024-09-11 06:20:37.559537] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63889190) [2024-09-11 06:20:37.560278] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5032, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.560446] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035637559550000, start_id=1726035637559550000, end_id=1726035637559550001) [2024-09-11 06:20:37.560907] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.560919] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.560927] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.561583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561634] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.561647] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.561653] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.561661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561692] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.561696] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.561704] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.561709] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.561715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561729] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.561743] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.561747] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.561750] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.561754] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.561788] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637561760), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.561800] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.561807] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.561814] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:20:37.664301] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63784426) [2024-09-11 06:20:37.665139] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5033, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.665304] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035637664802000, start_id=1726035637664802000, end_id=1726035637664802001) [2024-09-11 06:20:37.665699] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.665712] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.665720] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.666412] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666442] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666457] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.666470] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.666476] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.666482] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666494] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666505] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.666508] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.666514] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.666517] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.666520] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.666541] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.666544] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.666546] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.666548] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.666571] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637666551), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.666579] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.666587] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.666593] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2295, ddl_arg=NULL) [2024-09-11 06:20:37.769064] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63679661) [2024-09-11 06:20:37.769845] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5034, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.770015] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035637770008000, start_id=1726035637770008000, end_id=1726035637770008001) [2024-09-11 06:20:37.770455] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.770468] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.770477] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.771132] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771163] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771178] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.771190] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.771196] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.771202] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771214] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771225] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.771227] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.771231] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.771236] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.771239] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.771260] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.771263] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.771265] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.771267] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.771298] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637771271), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.771307] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.771314] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.771321] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:20:37.873631] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63575094) [2024-09-11 06:20:37.874435] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5035, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.874609] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035637874187000, start_id=1726035637874187000, end_id=1726035637874187001) [2024-09-11 06:20:37.875024] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.875037] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.875045] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.875677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875707] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875727] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.875742] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.875748] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.875755] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875765] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875787] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.875790] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.875793] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.875796] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.875800] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875811] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.875820] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:37.875823] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.875824] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.875827] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:37.875861] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637875829), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.875868] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.875875] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:37.875880] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2250, ddl_arg=NULL) [2024-09-11 06:20:37.978230] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63470519) [2024-09-11 06:20:37.979052] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5036, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:37.979220] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035637978350000, start_id=1726035637978350000, end_id=1726035637978350001) [2024-09-11 06:20:37.979662] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:37.979675] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:37.979684] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.980354] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.980385] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.980401] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:37.980412] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:37.980419] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:37.980425] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.980436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.980448] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:37.980450] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:37.980454] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.980456] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:37.980459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:37.980470] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:37.980482] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:37.980507] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035637980485), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:37.980515] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:37.980522] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.093147] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63355590) [2024-09-11 06:20:38.093965] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5037, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.094134] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035638093078000, start_id=1726035638093078000, end_id=1726035638093078001) [2024-09-11 06:20:38.094565] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:38.094578] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.094596] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.094605] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.095300] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095333] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.095362] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.095369] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.095377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095400] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.095403] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.095406] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.095413] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.095415] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095424] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.095436] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.095439] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.095441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.095443] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.095470] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638095446), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.095480] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.095487] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.095491] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2345, ddl_arg=NULL) [2024-09-11 06:20:38.197829] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63250924) [2024-09-11 06:20:38.198613] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5038, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.198781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035638198309000, start_id=1726035638198309000, end_id=1726035638198309001) [2024-09-11 06:20:38.199214] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.199229] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.199240] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.199933] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.199972] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.199989] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.200000] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.200005] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.200011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.200028] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.200042] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.200046] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.200050] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.200053] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.200056] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.200065] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.200075] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.200078] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.200081] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.200083] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.200106] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638200086), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.200115] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.200121] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.200128] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2301, ddl_arg=NULL) [2024-09-11 06:20:38.312916] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:38.314400] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.314415] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.314425] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.315128] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.315140] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.315165] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638315144), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.315175] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.315181] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.418315] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=63030441) [2024-09-11 06:20:38.419121] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5040, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.419334] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035638418306000, start_id=1726035638418306000, end_id=1726035638418306001) [2024-09-11 06:20:38.419759] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.419771] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:38.420459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420487] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420503] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.420513] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.420519] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.420525] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420536] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420547] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.420551] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.420557] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.420561] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.420564] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.420587] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.420590] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.420593] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.420596] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.420625] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638420602), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.420636] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.420645] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2332, ddl_arg=NULL) [2024-09-11 06:20:38.533509] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62915224) [2024-09-11 06:20:38.534281] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5041, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.534441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035638533958000, start_id=1726035638533958000, end_id=1726035638533958001) [2024-09-11 06:20:38.534886] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.534896] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.534902] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.535549] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535579] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535594] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.535607] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.535613] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.535618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535640] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.535647] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.535651] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.535659] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.535662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535673] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.535681] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.535683] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.535685] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.535688] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.535709] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638535691), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.535716] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.535724] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.535728] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2221, ddl_arg=NULL) [2024-09-11 06:20:38.638260] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62810470) [2024-09-11 06:20:38.639066] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5042, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.639233] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035638639174000, start_id=1726035638639174000, end_id=1726035638639174001) [2024-09-11 06:20:38.639661] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.639674] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.639682] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.640340] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640371] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640389] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.640400] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.640406] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.640411] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640422] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640434] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.640437] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.640441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.640444] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.640447] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640458] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.640467] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.640470] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.640472] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.640475] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.640497] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638640478), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.640506] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.640512] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.640516] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:20:38.753646] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62695101) [2024-09-11 06:20:38.754373] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5043, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.754534] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035638753974000, start_id=1726035638753974000, end_id=1726035638753974001) [2024-09-11 06:20:38.754958] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.754970] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.754979] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.755610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755638] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755655] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.755668] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.755682] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.755689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755701] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755713] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.755717] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.755720] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.755724] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.755730] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755742] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.755751] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.755754] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.755756] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.755760] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.755791] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638755766), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.755800] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.755805] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.755812] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2167, ddl_arg=NULL) [2024-09-11 06:20:38.858078] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62590656) [2024-09-11 06:20:38.858818] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5044, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.858968] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:38.859375] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.859387] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.859395] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.860046] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.860057] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.860077] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638860062), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.860086] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.860094] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.973132] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62475614) [2024-09-11 06:20:38.973976] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5045, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:38.974135] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035638973733000, start_id=1726035638973733000, end_id=1726035638973733001) [2024-09-11 06:20:38.974574] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:38.974583] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:38.974589] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.975266] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975300] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637065352, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975314] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:38.975326] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:38.975332] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:38.975339] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975350] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975361] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:38.975364] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.975369] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.975372] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:38.975376] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975387] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:38.975398] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:38.975400] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:38.975402] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:38.975404] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:38.975427] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035638975409), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:38.975435] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:38.975440] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:38.975445] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2315, ddl_arg=NULL) [2024-09-11 06:20:39.077908] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62370834) [2024-09-11 06:20:39.079064] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5046, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.079305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035639078995000, start_id=1726035639078995000, end_id=1726035639078995001) [2024-09-11 06:20:39.079753] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.079769] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.079810] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.080747] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080793] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=45] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080815] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.080830] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.080841] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.080848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080870] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080887] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.080894] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.080901] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.080911] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.080917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080935] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.080953] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.080958] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.080963] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.080969] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.080999] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639080975), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.081009] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.081018] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.081026] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=3121, ddl_arg=NULL) [2024-09-11 06:20:39.192681] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62256052) [2024-09-11 06:20:39.193416] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5047, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.193592] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035639192585000, start_id=1726035639192585000, end_id=1726035639192585001) [2024-09-11 06:20:39.194019] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:39.194031] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.194040] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.194047] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.194733] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194760] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194784] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.194800] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.194808] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.194817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637207846, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194849] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.194852] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.194855] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.194858] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.194863] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194875] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035637276444, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.194885] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.194887] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.194889] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.194892] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.194911] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639194895), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.194918] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.194925] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.194928] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2250, ddl_arg=NULL) [2024-09-11 06:20:39.297215] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62151515) [2024-09-11 06:20:39.298052] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5048, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.298211] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035639297890000, start_id=1726035639297890000, end_id=1726035639297890001) [2024-09-11 06:20:39.298655] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.298668] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.298675] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.299362] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299391] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299408] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.299419] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.299426] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.299430] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299442] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299455] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.299458] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.299462] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.299468] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.299472] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.299497] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.299500] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.299505] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.299509] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.299531] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639299513), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.299540] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.299547] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.299552] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2339, ddl_arg=NULL) [2024-09-11 06:20:39.402215] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=62046513) [2024-09-11 06:20:39.402967] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.403135] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035639403003000, start_id=1726035639403003000, end_id=1726035639403003001) [2024-09-11 06:20:39.403653] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.403662] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.403674] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.404353] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.404384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.404400] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.404416] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.404424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:39.404446] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639404428), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.404455] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.404461] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.506865] INFO [RS] add_server (ob_root_service.cpp:7915) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.508401] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.508414] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.508425] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.509078] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.509107] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.509155] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639509139), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.509163] INFO [RS] add_server (ob_root_service.cpp:7928) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.509172] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.611347] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61837372) [2024-09-11 06:20:39.612153] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5051, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.612335] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035639611305000, start_id=1726035639611305000, end_id=1726035639611305001) [2024-09-11 06:20:39.612747] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.612760] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.612769] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.613422] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613454] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613468] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.613479] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.613482] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.613487] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613516] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.613518] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.613522] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.613528] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.613531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613542] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.613552] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.613557] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.613559] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.613584] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639613563), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.613592] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.613600] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.613608] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2263, ddl_arg=NULL) [2024-09-11 06:20:39.715931] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61732829) [2024-09-11 06:20:39.716705] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5052, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.716933] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035639716196000, start_id=1726035639716196000, end_id=1726035639716196001) [2024-09-11 06:20:39.717357] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.717371] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.717381] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.718115] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718148] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718164] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.718180] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.718186] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.718192] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718209] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718221] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.718223] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.718229] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.718234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.718238] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.718262] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.718265] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.718268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.718295] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639718272), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.718303] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.718310] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.718317] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2388, ddl_arg=NULL) [2024-09-11 06:20:39.820506] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61628220) [2024-09-11 06:20:39.821327] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5053, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.821497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035639821299000, start_id=1726035639821299000, end_id=1726035639821299001) [2024-09-11 06:20:39.821925] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.821937] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.821944] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.822645] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822683] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822699] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.822709] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.822714] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.822721] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822734] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822745] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.822748] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.822752] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.822758] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.822761] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822770] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.822817] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=47] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.822819] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.822822] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.822843] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639822825), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.822850] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.822858] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:39.822861] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2357, ddl_arg=NULL) [2024-09-11 06:20:39.924931] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61523805) [2024-09-11 06:20:39.925695] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5054, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:39.925876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035639925433000, start_id=1726035639925433000, end_id=1726035639925433001) [2024-09-11 06:20:39.926299] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:39.926314] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:39.926324] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.927001] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927047] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:39.927058] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:39.927064] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:39.927069] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927081] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927092] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:39.927095] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.927099] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:39.927105] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:39.927108] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927119] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:39.927130] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:39.927133] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:39.927135] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:39.927158] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035639927139), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:39.927167] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:39.927175] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.029508] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61419218) [2024-09-11 06:20:40.030268] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5055, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.030429] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035640029505000, start_id=1726035640029505000, end_id=1726035640029505001) [2024-09-11 06:20:40.030835] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.030848] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.030859] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.031553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.031583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.031599] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.031609] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.031614] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.031622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.031633] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.031643] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.031646] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.031652] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.031656] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.031660] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.031693] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640031672), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.031701] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.031707] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.134162] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61314565) [2024-09-11 06:20:40.134994] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5056, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.135171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035640134723000, start_id=1726035640134723000, end_id=1726035640134723001) [2024-09-11 06:20:40.135594] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.135608] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.135617] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.136285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136316] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136330] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.136341] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.136347] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.136353] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136364] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136376] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.136379] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.136384] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.136388] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136398] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.136409] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:40.136413] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.136416] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:40.136442] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640136420), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.136452] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.136462] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.136470] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2311, ddl_arg=NULL) [2024-09-11 06:20:40.238530] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61210201) [2024-09-11 06:20:40.239357] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5057, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.239542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035640238833000, start_id=1726035640238833000, end_id=1726035640238833001) [2024-09-11 06:20:40.239986] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:40.239996] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.240004] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.240011] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.240656] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240688] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240704] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.240717] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.240724] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.240729] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240740] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240751] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.240754] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.240760] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.240765] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.240772] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240797] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.240812] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:40.240815] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.240817] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.240819] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:40.240841] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640240822), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.240850] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.240860] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.240866] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2338, ddl_arg=NULL) [2024-09-11 06:20:40.343517] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61105205) [2024-09-11 06:20:40.344296] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5058, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.344462] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035640344175000, start_id=1726035640344175000, end_id=1726035640344175001) [2024-09-11 06:20:40.344527] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=268, avg_log_batch_cnt=1, total_group_log_size=182062, avg_group_log_size=679) [2024-09-11 06:20:40.344539] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=181873) [2024-09-11 06:20:40.344907] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.344920] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.344928] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.345553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.345581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.345621] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.345630] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.345652] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640345634), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.345658] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.345665] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.448685] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=61000040) [2024-09-11 06:20:40.449469] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5059, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.449638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035640449244000, start_id=1726035640449244000, end_id=1726035640449244001) [2024-09-11 06:20:40.450088] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.450103] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.450110] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.450833] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.450858] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.450872] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.450886] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.450892] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.450897] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.450910] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.450915] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.450938] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640450921), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.450946] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.450952] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.554382] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60894359) [2024-09-11 06:20:40.555212] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5060, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.555392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035640554592000, start_id=1726035640554592000, end_id=1726035640554592001) [2024-09-11 06:20:40.555857] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.556493] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.556523] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.556539] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.556555] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.556561] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.556567] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.556578] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.556602] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.556607] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.556611] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.556615] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.556620] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.556646] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640556625), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.556654] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.661442] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60787284) [2024-09-11 06:20:40.662277] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5061, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.662439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035640661779000, start_id=1726035640661779000, end_id=1726035640661779001) [2024-09-11 06:20:40.662844] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.663489] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663520] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663536] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.663546] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.663552] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.663560] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663583] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.663586] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.663592] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.663595] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.663598] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663608] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.663620] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.663643] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640663624), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.663650] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.766485] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60682246) [2024-09-11 06:20:40.767231] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5062, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.767400] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035640767007000, start_id=1726035640767007000, end_id=1726035640767007001) [2024-09-11 06:20:40.767811] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.767824] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.767832] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.768451] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.768484] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.768498] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.768513] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.768521] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.768541] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640768524), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.768549] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.768556] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.871495] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60577231) [2024-09-11 06:20:40.872312] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5063, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.872493] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035640872160000, start_id=1726035640872160000, end_id=1726035640872160001) [2024-09-11 06:20:40.872925] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.872938] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.872947] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.873623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.873652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.873668] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.873680] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.873685] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.873692] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.873704] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.873715] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.873718] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.873721] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.873727] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.873730] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:40.873743] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.873766] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640873746), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.873783] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.873792] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.977002] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60471731) [2024-09-11 06:20:40.977794] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5064, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:40.977965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035640977501000, start_id=1726035640977501000, end_id=1726035640977501001) [2024-09-11 06:20:40.978412] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:40.978426] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:40.978436] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.979098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639065739, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979143] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:40.979156] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:40.979162] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:40.979169] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979181] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979192] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:40.979195] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.979199] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.979204] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:40.979208] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979219] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:40.979230] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:40.979233] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:40.979235] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:40.979237] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:40.979262] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035640979241), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:40.979270] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:40.979278] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:40.979284] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2284, ddl_arg=NULL) [2024-09-11 06:20:41.082192] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60366546) [2024-09-11 06:20:41.083105] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5065, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.083340] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035641082700000, start_id=1726035641082700000, end_id=1726035641082700001) [2024-09-11 06:20:41.083830] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.083843] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.083849] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.084597] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084645] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.084655] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.084666] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.084672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084684] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084696] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:41.084699] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.084703] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.084707] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:41.084710] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084722] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639276895, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.084734] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:41.084737] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.084739] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.084743] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:41.084766] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641084746), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.084798] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.084807] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.084811] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2621, ddl_arg=NULL) [2024-09-11 06:20:41.187482] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60261261) [2024-09-11 06:20:41.188278] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5066, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.188464] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035641188008000, start_id=1726035641188008000, end_id=1726035641188008001) [2024-09-11 06:20:41.188904] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.188918] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.188926] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.189563] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.189591] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.189607] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.189620] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.189628] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.189636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.189649] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035639284832, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.189660] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:41.189664] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.189671] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.189696] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641189675), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.189704] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.189710] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.292689] INFO [RS] add_server (ob_root_service.cpp:7915) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=60156040) [2024-09-11 06:20:41.293581] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5067, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.293782] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035641293030000, start_id=1726035641293030000, end_id=1726035641293030001) [2024-09-11 06:20:41.294219] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:41.294234] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.294242] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.294249] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.294926] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.294956] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.294973] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:41.294986] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.294994] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.295015] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641294998), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.295023] INFO [RS] add_server (ob_root_service.cpp:7928) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.295030] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.397867] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:41.398660] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5068, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.398852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035641398264000, start_id=1726035641398264000, end_id=1726035641398264001) [2024-09-11 06:20:41.398919] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=34, avg_log_batch_cnt=1, total_group_log_size=28203, avg_group_log_size=829) [2024-09-11 06:20:41.398932] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28203) [2024-09-11 06:20:41.399287] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.399300] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.399308] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.399980] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.400010] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.400025] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.400037] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.400043] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.400050] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.400063] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.400067] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.400092] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641400070), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.400101] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.400107] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.503010] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59945726) [2024-09-11 06:20:41.503765] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5069, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.503987] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1726035641503648000, start_id=1726035641503648000, end_id=1726035641503648001) [2024-09-11 06:20:41.504414] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.504429] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.504438] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.505111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.505141] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.505164] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.505178] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.505186] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.505197] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.505204] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.505225] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641505207), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.505233] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.505239] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.607938] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59840793) [2024-09-11 06:20:41.608740] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5070, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.609006] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035641608834000, start_id=1726035641608834000, end_id=1726035641608834001) [2024-09-11 06:20:41.609432] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.609445] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.609458] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.610182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610210] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610227] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.610237] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.610243] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.610248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610262] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610278] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:41.610283] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.610288] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.610295] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:41.610299] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610315] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.610332] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:41.610335] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.610337] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.610341] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:41.610363] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641610344), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.610371] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.610377] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.610381] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2445, ddl_arg=NULL) [2024-09-11 06:20:41.712950] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59735791) [2024-09-11 06:20:41.713896] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5071, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.714167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035641714141000, start_id=1726035641714141000, end_id=1726035641714141001) [2024-09-11 06:20:41.714619] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.714635] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:41.715564] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.715579] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.715607] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641715587), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.715620] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.818465] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59630284) [2024-09-11 06:20:41.819225] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5072, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.819402] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035641818425000, start_id=1726035641818425000, end_id=1726035641818425001) [2024-09-11 06:20:41.819843] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.819857] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.819864] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.820513] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820556] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.820566] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.820573] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.820578] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820589] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820600] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:41.820603] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.820607] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.820613] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:41.820616] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.820638] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:41.820641] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:41.820646] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.820649] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:41.820676] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641820654), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.820686] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.820692] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:41.820697] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2233, ddl_arg=NULL) [2024-09-11 06:20:41.933757] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59514995) [2024-09-11 06:20:41.934543] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5073, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:41.934708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035641934358000, start_id=1726035641934358000, end_id=1726035641934358001) [2024-09-11 06:20:41.935140] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:41.935151] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:41.935158] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.935834] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.935865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.935882] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:41.935893] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:41.935900] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:41.935904] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.935917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:41.935928] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:41.935931] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:41.935935] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.935939] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:41.935961] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035641935944), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:41.935968] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:41.935973] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.039308] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59409417) [2024-09-11 06:20:42.040136] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5074, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.040315] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035642039738000, start_id=1726035642039738000, end_id=1726035642039738001) [2024-09-11 06:20:42.040729] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.040743] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.040752] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.041429] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041474] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.041484] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.041491] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.041496] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041507] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041519] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.041521] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.041525] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.041529] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.041534] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041544] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.041559] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.041564] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.041567] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.041570] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.041593] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642041573), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.041601] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.041608] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.041614] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2309, ddl_arg=NULL) [2024-09-11 06:20:42.144267] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59304480) [2024-09-11 06:20:42.145001] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5075, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.145163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035642144887000, start_id=1726035642144887000, end_id=1726035642144887001) [2024-09-11 06:20:42.145576] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.145588] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.145596] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.146254] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146283] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146300] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.146310] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.146317] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.146322] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146334] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.146354] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.146357] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.146361] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.146364] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146376] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.146388] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.146391] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.146393] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.146397] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.146418] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642146401), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.146426] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.146434] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.146438] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2172, ddl_arg=NULL) [2024-09-11 06:20:42.248952] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=59199780) [2024-09-11 06:20:42.249699] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5076, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.249865] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035642249010000, start_id=1726035642249010000, end_id=1726035642249010001) [2024-09-11 06:20:42.250270] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.250281] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.250288] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.250952] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.250984] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.251001] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.251012] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.251018] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.251025] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.251035] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.251045] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.251048] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.251052] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.251055] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.251061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.251072] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.251083] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.251087] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.251094] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.251107] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.251133] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642251112), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.251141] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.251148] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.251152] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2203, ddl_arg=NULL) [2024-09-11 06:20:42.364188] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:42.364996] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5077, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.365636] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:42.365650] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.365661] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.365672] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.366399] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.366440] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.366451] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.366457] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.366482] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642366463), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.366491] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.366499] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.469230] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58979501) [2024-09-11 06:20:42.470055] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5078, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.470236] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035642470159000, start_id=1726035642470159000, end_id=1726035642470159001) [2024-09-11 06:20:42.470651] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.470664] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.470678] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.471376] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471411] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471427] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.471440] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.471446] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.471450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471462] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471474] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.471476] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.471482] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.471485] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.471491] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.471514] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.471516] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.471518] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.471522] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.471553] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642471525), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.471562] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.471569] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.471574] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2346, ddl_arg=NULL) [2024-09-11 06:20:42.573909] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58874820) [2024-09-11 06:20:42.574667] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5079, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.574859] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035642574288000, start_id=1726035642574288000, end_id=1726035642574288001) [2024-09-11 06:20:42.575280] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.575293] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.575302] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.575958] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.575988] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.576005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.576016] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.576020] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.576027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.576038] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.576050] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.576053] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.576057] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.576061] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.576067] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.576079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.576095] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.576099] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.576102] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.576110] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.576132] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642576114), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.576140] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.576146] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.576152] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2245, ddl_arg=NULL) [2024-09-11 06:20:42.678502] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58770233) [2024-09-11 06:20:42.679286] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5080, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.679439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035642679397000, start_id=1726035642679397000, end_id=1726035642679397001) [2024-09-11 06:20:42.679857] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.679869] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.679878] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.680499] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680528] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680543] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.680553] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.680559] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.680563] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680585] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.680588] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.680592] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.680599] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.680602] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.680625] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.680627] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.680630] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.680633] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.680652] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642680636), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.680658] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.680663] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.680667] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2167, ddl_arg=NULL) [2024-09-11 06:20:42.783619] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58665119) [2024-09-11 06:20:42.784441] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5081, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.784614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035642784597000, start_id=1726035642784597000, end_id=1726035642784597001) [2024-09-11 06:20:42.785071] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.785084] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:42.785735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785766] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785787] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.785798] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.785803] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.785808] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785821] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785830] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.785833] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.785838] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.785844] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.785848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785860] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.785872] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.785874] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.785877] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.785879] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.785904] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642785883), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.785911] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.785917] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2300, ddl_arg=NULL) [2024-09-11 06:20:42.888288] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58560462) [2024-09-11 06:20:42.889080] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5082, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:42.889328] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:157) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] submit log success(service_type_=0, last_id=1726035659537832000, limited_id=1726035659537832000) [2024-09-11 06:20:42.889338] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] submit log with lock success(service_type_=0, last_id=1726035659537832000, limited_id=1726035659537832000) [2024-09-11 06:20:42.889342] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1726035642888742000, start_id=1726035642888742000, end_id=1726035642888742001) [2024-09-11 06:20:42.889847] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:42.889857] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:42.889865] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.890502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890547] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:42.890559] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:42.890566] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:42.890572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890595] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:42.890599] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.890604] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.890608] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:42.890612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:42.890639] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:42.890642] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:42.890644] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:42.890646] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:42.890669] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035642890650), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:42.890677] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:42.890684] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:42.890690] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2404, ddl_arg=NULL) [2024-09-11 06:20:43.003350] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58445405) [2024-09-11 06:20:43.004380] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5083, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.004614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1726035643004539000, start_id=1726035643004539000, end_id=1726035643004539001) [2024-09-11 06:20:43.005078] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.005093] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.005101] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.005750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005792] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641066080, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005812] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.005824] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.005831] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.005836] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005860] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.005863] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.005869] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.005873] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.005877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005889] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.005905] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.005908] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.005911] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.005914] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.005941] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643005920), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.005950] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.005957] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.005963] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2615, ddl_arg=NULL) [2024-09-11 06:20:43.108297] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58340450) [2024-09-11 06:20:43.109092] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5084, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.109261] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035643108765000, start_id=1726035643108765000, end_id=1726035643108765001) [2024-09-11 06:20:43.109718] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.109732] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.109741] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.110510] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110545] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110559] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.110572] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.110578] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.110583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110594] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110606] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.110608] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.110613] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.110617] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.110622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110633] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.110643] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.110646] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.110649] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.110651] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.110677] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643110655), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.110687] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.110699] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.110706] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2411, ddl_arg=NULL) [2024-09-11 06:20:43.223349] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58225389) [2024-09-11 06:20:43.224172] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5085, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.224348] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035643223598000, start_id=1726035643223598000, end_id=1726035643223598001) [2024-09-11 06:20:43.224824] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.224838] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.224847] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.225492] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225545] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.225555] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.225561] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.225566] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225577] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641285657, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225589] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.225592] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.225596] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.225607] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.225610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035641277340, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.225633] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.225636] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.225639] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.225641] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.225678] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643225644), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.225688] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.225697] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.225701] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2354, ddl_arg=NULL) [2024-09-11 06:20:43.329115] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5086, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.329289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035643328718000, start_id=1726035643328718000, end_id=1726035643328718001) [2024-09-11 06:20:43.329696] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.329710] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.329720] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.330382] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.330412] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.330427] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.330441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.330449] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.330473] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643330453), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.330482] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.330489] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.433181] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=58015577) [2024-09-11 06:20:43.433965] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5087, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.434143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1726035643433901000, start_id=1726035643433901000, end_id=1726035643433901001) [2024-09-11 06:20:43.434198] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=27234, avg_group_log_size=778) [2024-09-11 06:20:43.434211] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=27004) [2024-09-11 06:20:43.434542] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:43.434550] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.434557] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.434564] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.435238] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435268] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435283] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.435296] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.435302] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.435307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435319] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435330] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.435333] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.435337] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.435343] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.435346] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435358] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.435369] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.435372] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.435375] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.435378] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.435408] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643435384), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.435417] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.435425] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.435429] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2250, ddl_arg=NULL) [2024-09-11 06:20:43.537861] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57910861) [2024-09-11 06:20:43.538638] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5088, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.538817] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035643537783000, start_id=1726035643537783000, end_id=1726035643537783001) [2024-09-11 06:20:43.539208] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.539220] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.539229] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.539878] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.539909] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.539925] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.539936] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.539942] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.539947] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.539959] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.539971] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.539974] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.539978] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.539982] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.539985] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.539997] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.540007] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.540011] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.540013] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.540016] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.540041] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643540019), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.540050] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.540057] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.540063] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2204, ddl_arg=NULL) [2024-09-11 06:20:43.642207] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57806523) [2024-09-11 06:20:43.643038] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5089, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.643218] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035643642832000, start_id=1726035643642832000, end_id=1726035643642832001) [2024-09-11 06:20:43.643623] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.643636] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.643642] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.644304] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.644335] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.644349] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.644359] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:43.644368] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.644374] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.644399] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643644380), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.644407] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.644415] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.746918] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57701832) [2024-09-11 06:20:43.747651] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5090, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.747837] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035643746944000, start_id=1726035643746944000, end_id=1726035643746944001) [2024-09-11 06:20:43.748252] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.748267] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.748278] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.748951] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.748987] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.749002] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.749014] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.749018] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.749023] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.749035] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.749046] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.749049] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.749054] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.749058] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.749063] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.749074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.749084] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.749087] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.749089] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.749091] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.749114] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643749095), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.749122] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.749132] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.749140] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2224, ddl_arg=NULL) [2024-09-11 06:20:43.851494] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57597239) [2024-09-11 06:20:43.852322] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5091, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.852486] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035643852210000, start_id=1726035643852210000, end_id=1726035643852210001) [2024-09-11 06:20:43.852875] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:43.852886] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:43.852894] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.853523] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.853551] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.853566] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.853577] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.853583] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.853589] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:43.853601] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.853608] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.853627] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643853611), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.853635] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.853642] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:43.956461] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57492270) [2024-09-11 06:20:43.957292] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5092, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:43.957450] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035643957401000, start_id=1726035643957401000, end_id=1726035643957401001) [2024-09-11 06:20:43.958492] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958521] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958538] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:43.958548] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:43.958554] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:43.958559] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958570] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958581] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:43.958584] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.958588] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.958595] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:43.958599] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958611] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:43.958621] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:43.958627] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:43.958629] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:43.958633] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:43.958654] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035643958637), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:43.958663] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:43.958672] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:20:44.061132] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57387605) [2024-09-11 06:20:44.061882] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5093, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.062049] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035644061625000, start_id=1726035644061625000, end_id=1726035644061625001) [2024-09-11 06:20:44.062465] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.062477] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.062484] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.063177] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063207] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063223] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.063236] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.063241] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.063248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063259] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063270] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.063273] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.063278] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.063281] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.063283] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063295] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.063305] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.063308] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.063310] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.063312] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.063332] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644063315), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.063340] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.063347] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.063353] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2223, ddl_arg=NULL) [2024-09-11 06:20:44.165600] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57283129) [2024-09-11 06:20:44.166388] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5094, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.166554] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1726035644165776000, start_id=1726035644165776000, end_id=1726035644165776001) [2024-09-11 06:20:44.167025] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.167043] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.167052] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.167934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.167962] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.167977] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.167988] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.167994] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.168003] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.168021] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.168032] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.168035] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.168038] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.168041] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.168044] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.168054] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.168065] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.168067] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.168069] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.168072] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.168091] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644168075), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.168099] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.168106] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.168110] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2512, ddl_arg=NULL) [2024-09-11 06:20:44.270273] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57178463) [2024-09-11 06:20:44.271061] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5095, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.271222] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035644270913000, start_id=1726035644270913000, end_id=1726035644270913001) [2024-09-11 06:20:44.271659] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.271676] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.271685] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.272377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272406] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272422] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.272432] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.272440] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.272445] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272472] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.272475] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.272480] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.272483] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.272487] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272498] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.272507] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.272511] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.272516] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.272519] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.272540] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644272523), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.272549] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.272559] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.272563] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2292, ddl_arg=NULL) [2024-09-11 06:20:44.374949] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=57073802) [2024-09-11 06:20:44.375732] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5096, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.375917] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035644375147000, start_id=1726035644375147000, end_id=1726035644375147001) [2024-09-11 06:20:44.376330] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.376342] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.376350] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.377059] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:44.377096] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.377105] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.377127] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644377109), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.377133] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.377139] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.479720] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56969014) [2024-09-11 06:20:44.480513] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5097, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.480684] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035644480313000, start_id=1726035644480313000, end_id=1726035644480313001) [2024-09-11 06:20:44.480744] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28259, avg_group_log_size=807) [2024-09-11 06:20:44.480758] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28259) [2024-09-11 06:20:44.481118] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:44.481127] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.481135] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.481145] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.481804] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481833] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481848] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.481860] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.481866] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.481872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481884] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481895] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.481898] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.481901] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.481907] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.481911] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.481933] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.481935] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.481937] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.481940] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.481966] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644481945), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.481976] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.481982] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.481987] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2270, ddl_arg=NULL) [2024-09-11 06:20:44.584651] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56864080) [2024-09-11 06:20:44.585484] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5098, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.585663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035644585560000, start_id=1726035644585560000, end_id=1726035644585560001) [2024-09-11 06:20:44.586117] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.586134] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.586145] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.587007] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587038] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587053] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.587066] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.587071] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.587078] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587090] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587099] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.587102] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.587108] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.587111] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.587116] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587126] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.587137] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.587139] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.587143] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.587145] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.587170] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644587149), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.587179] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.587187] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.587193] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2544, ddl_arg=NULL) [2024-09-11 06:20:44.689271] INFO [RS] add_server (ob_root_service.cpp:7915) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56759455) [2024-09-11 06:20:44.690127] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5099, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.690305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035644689699000, start_id=1726035644689699000, end_id=1726035644689699001) [2024-09-11 06:20:44.690736] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.690749] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.690758] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.691475] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691503] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691519] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.691529] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.691536] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.691542] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691565] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.691568] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.691572] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.691578] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.691581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.691603] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.691607] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.691610] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.691616] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.691637] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644691619), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.691645] INFO [RS] add_server (ob_root_service.cpp:7928) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.691651] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.691656] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55209][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2387, ddl_arg=NULL) [2024-09-11 06:20:44.793809] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56654935) [2024-09-11 06:20:44.794603] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5100, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.794761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035644793790000, start_id=1726035644793790000, end_id=1726035644793790001) [2024-09-11 06:20:44.795205] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.795218] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.795228] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.795895] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.795926] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.795940] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.795949] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.795954] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.795963] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.795976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.795988] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.795991] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.795996] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.795999] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.796002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.796016] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.796033] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.796037] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.796040] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.796042] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:44.796064] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644796047), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.796072] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.796079] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:44.796084] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2277, ddl_arg=NULL) [2024-09-11 06:20:44.898882] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56549848) [2024-09-11 06:20:44.899637] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5101, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:44.899839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035644898960000, start_id=1726035644898960000, end_id=1726035644898960001) [2024-09-11 06:20:44.900256] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:44.900266] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:44.900273] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.900942] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.900969] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.900984] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:44.900996] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:44.901001] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:44.901006] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.901017] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.901027] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:44.901029] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:44.901035] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.901039] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:44.901042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.901053] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:44.901064] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:44.901067] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:44.901086] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035644901070), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:44.901093] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:44.901100] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.013865] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56434868) [2024-09-11 06:20:45.014645] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5102, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.014834] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035645014780000, start_id=1726035645014780000, end_id=1726035645014780001) [2024-09-11 06:20:45.015310] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.015322] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.015327] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.015956] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.015985] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643066433, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.016000] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.016010] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.016022] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.016026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.016039] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.016050] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.016053] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.016056] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.016058] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.016062] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.016071] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.016082] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.016084] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.016087] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.016089] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.016107] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645016092), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.016115] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.016123] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.016127] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2264, ddl_arg=NULL) [2024-09-11 06:20:45.118589] INFO [RS] add_server (ob_root_service.cpp:7915) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56330154) [2024-09-11 06:20:45.119423] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5103, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.119604] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035645119126000, start_id=1726035645119126000, end_id=1726035645119126001) [2024-09-11 06:20:45.120061] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.120075] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.120084] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.120712] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120740] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.120766] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.120782] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.120789] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120808] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120826] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.120830] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.120834] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.120842] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.120847] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120864] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.120881] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.120885] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.120889] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.120893] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.120927] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645120908), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.120937] INFO [RS] add_server (ob_root_service.cpp:7928) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.120947] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.120954] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2367, ddl_arg=NULL) [2024-09-11 06:20:45.233452] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56215284) [2024-09-11 06:20:45.234288] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5104, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.234453] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035645233656000, start_id=1726035645233656000, end_id=1726035645233656001) [2024-09-11 06:20:45.234896] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.234911] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.234920] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.235572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235602] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235617] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.235629] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.235636] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.235643] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235655] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643286204, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235666] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.235669] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.235675] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.235678] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.235683] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235694] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035643277748, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.235703] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.235705] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.235707] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.235712] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.235738] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645235717), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.235745] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.235752] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.235758] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2309, ddl_arg=NULL) [2024-09-11 06:20:45.337696] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56111040) [2024-09-11 06:20:45.338455] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5105, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.338626] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035645337780000, start_id=1726035645337780000, end_id=1726035645337780001) [2024-09-11 06:20:45.339066] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.339079] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.339087] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.339714] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339743] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339757] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.339769] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.339789] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.339797] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339813] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339824] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.339826] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.339830] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.339834] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.339837] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.339866] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.339869] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.339871] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.339873] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.339893] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645339876), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.339900] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.339907] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.339914] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2220, ddl_arg=NULL) [2024-09-11 06:20:45.442828] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=56005914) [2024-09-11 06:20:45.443588] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5106, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.443753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035645443003000, start_id=1726035645443003000, end_id=1726035645443003001) [2024-09-11 06:20:45.444278] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.444290] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.444300] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.444974] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445003] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445019] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.445029] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.445044] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.445049] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445062] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445075] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.445078] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.445081] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.445086] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.445091] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445103] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.445113] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.445115] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.445118] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.445123] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.445144] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645445127), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.445152] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.445158] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.445165] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2338, ddl_arg=NULL) [2024-09-11 06:20:45.547703] INFO [RS] add_server (ob_root_service.cpp:7915) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55901025) [2024-09-11 06:20:45.548537] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5107, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.548701] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035645548234000, start_id=1726035645548234000, end_id=1726035645548234001) [2024-09-11 06:20:45.549131] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:45.549142] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.549149] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.549158] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.549819] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549871] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.549881] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.549890] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.549898] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549915] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549931] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.549935] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.549940] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.549945] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.549953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549972] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.549988] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.549993] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.550001] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.550005] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.550035] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645550014), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.550042] INFO [RS] add_server (ob_root_service.cpp:7928) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.550053] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.550063] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:45.662940] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55785781) [2024-09-11 06:20:45.663677] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5108, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.663844] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035645663065000, start_id=1726035645663065000, end_id=1726035645663065001) [2024-09-11 06:20:45.664300] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.664313] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.664321] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.664950] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.664986] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.665003] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.665015] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.665021] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.665027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.665040] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.665051] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.665053] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.665056] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.665059] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.665062] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.665073] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.665087] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.665090] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.665094] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.665097] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.665120] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645665104), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.665127] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.665135] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.665138] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2200, ddl_arg=NULL) [2024-09-11 06:20:45.796137] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55652593) [2024-09-11 06:20:45.796945] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5109, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.797134] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035645796710000, start_id=1726035645796710000, end_id=1726035645796710001) [2024-09-11 06:20:45.797682] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.797696] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.797702] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.798494] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798521] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798540] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.798556] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.798564] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.798570] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798587] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798597] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.798599] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.798607] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.798610] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.798614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798638] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.798702] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=64] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.798706] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.798708] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.798711] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.798732] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645798715), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.798740] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.798749] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.798752] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2617, ddl_arg=NULL) [2024-09-11 06:20:45.901284] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55547449) [2024-09-11 06:20:45.902094] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5110, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:45.902290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035645901949000, start_id=1726035645901949000, end_id=1726035645901949001) [2024-09-11 06:20:45.902755] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:45.902768] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:45.902787] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.903420] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903449] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903464] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:45.903475] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:45.903481] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:45.903485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903501] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903515] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:45.903518] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.903522] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.903526] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:45.903533] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903549] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:45.903561] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:45.903564] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:45.903567] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:45.903569] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:45.903593] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035645903575), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:45.903610] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:45.903616] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:45.903622] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2339, ddl_arg=NULL) [2024-09-11 06:20:46.007910] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55440822) [2024-09-11 06:20:46.008657] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5111, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.008852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035646008180000, start_id=1726035646008180000, end_id=1726035646008180001) [2024-09-11 06:20:46.009293] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.009305] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.009311] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.009964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.009992] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.010011] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.010023] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.010028] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.010032] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.010048] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.010063] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.010065] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.010070] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.010073] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.010076] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.010086] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.010097] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.010100] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.010102] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.010104] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.010124] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646010107), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.010131] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.010139] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.010145] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2237, ddl_arg=NULL) [2024-09-11 06:20:46.123170] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55325559) [2024-09-11 06:20:46.124025] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5112, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.124206] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035646123793000, start_id=1726035646123793000, end_id=1726035646123793001) [2024-09-11 06:20:46.124647] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.124660] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.125316] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125346] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125360] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.125369] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.125376] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.125381] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125394] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125405] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.125409] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.125412] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.125415] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.125419] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125430] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.125440] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.125443] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.125445] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.125447] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.125473] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646125451), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.125481] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.125491] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2323, ddl_arg=NULL) [2024-09-11 06:20:46.228211] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55220515) [2024-09-11 06:20:46.228985] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5113, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.229146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035646229075000, start_id=1726035646229075000, end_id=1726035646229075001) [2024-09-11 06:20:46.229548] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.229560] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.229568] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.230285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230313] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230329] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.230341] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.230347] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.230352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230374] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.230377] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.230383] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.230388] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.230391] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230403] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.230412] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.230414] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.230417] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.230437] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646230420), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.230446] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.230452] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.230458] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2249, ddl_arg=NULL) [2024-09-11 06:20:46.343483] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55105244) [2024-09-11 06:20:46.344279] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5114, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.344435] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035646343556000, start_id=1726035646343556000, end_id=1726035646343556001) [2024-09-11 06:20:46.344906] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.344917] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.344923] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.345548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345592] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.345602] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.345609] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.345613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345625] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345637] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.345640] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.345644] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.345650] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.345653] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.345675] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.345678] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.345681] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.345699] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646345684), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.345708] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.345715] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.345721] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2239, ddl_arg=NULL) [2024-09-11 06:20:46.448682] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=55000063) [2024-09-11 06:20:46.449486] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5115, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.449655] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035646448848000, start_id=1726035646448848000, end_id=1726035646448848001) [2024-09-11 06:20:46.450027] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.450041] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.450050] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.450737] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450767] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450795] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.450808] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.450815] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.450820] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450832] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450844] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.450848] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.450851] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.450856] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.450860] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450871] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.450883] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.450886] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.450889] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.450914] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646450893), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.450923] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.450928] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.450935] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2255, ddl_arg=NULL) [2024-09-11 06:20:46.553962] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54894766) [2024-09-11 06:20:46.554812] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5116, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.554985] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035646554114000, start_id=1726035646554114000, end_id=1726035646554114001) [2024-09-11 06:20:46.555415] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:46.555425] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.555434] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.555449] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.556180] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556217] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=36] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556233] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.556243] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.556249] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.556255] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556266] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556278] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.556281] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.556287] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.556292] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556303] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.556313] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.556316] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.556320] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.556344] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646556324), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.556354] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.556361] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.556367] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2408, ddl_arg=NULL) [2024-09-11 06:20:46.658745] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54789980) [2024-09-11 06:20:46.659592] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5117, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.659761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035646659193000, start_id=1726035646659193000, end_id=1726035646659193001) [2024-09-11 06:20:46.660211] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.660225] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.660234] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.660934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.660964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.660980] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.660990] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.660997] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.661002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.661014] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.661025] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.661028] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.661032] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.661038] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.661042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.661055] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.661080] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646661059), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.661088] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.661094] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.773788] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54674939) [2024-09-11 06:20:46.774564] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5118, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.774725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.775176] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.775190] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.775198] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.775861] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.775890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.775905] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.775916] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.775924] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.775931] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.775954] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646775935), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.775962] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.775970] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.878527] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54570197) [2024-09-11 06:20:46.879349] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5119, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.879524] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035646879015000, start_id=1726035646879015000, end_id=1726035646879015001) [2024-09-11 06:20:46.879981] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.879995] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.880004] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.880661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880692] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880711] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.880723] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.880731] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.880736] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880747] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.880758] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.880764] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.880767] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.880800] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880816] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.880830] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.880838] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.880843] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:46.880849] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.880883] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646880857), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.880893] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.880903] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.880911] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2386, ddl_arg=NULL) [2024-09-11 06:20:46.993583] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54455156) [2024-09-11 06:20:46.994405] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5120, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:46.994577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035646993613000, start_id=1726035646993613000, end_id=1726035646993613001) [2024-09-11 06:20:46.995027] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:46.995040] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:46.995048] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.995723] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645066816, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7402344GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:46.995791] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] added async task to update rslist(force_update=false) [2024-09-11 06:20:46.995796] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:46.995803] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995815] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995827] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:46.995830] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.995834] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:46.995840] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:46.995843] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995854] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:46.995864] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:46.995867] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:46.995870] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:46.995896] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035646995873), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:46.995906] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:46.995913] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:46.995917] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2336, ddl_arg=NULL) [2024-09-11 06:20:47.098338] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54350390) [2024-09-11 06:20:47.099194] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5121, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.099365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035647098894000, start_id=1726035647098894000, end_id=1726035647098894001) [2024-09-11 06:20:47.099809] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.099821] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.099830] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.100478] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100509] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100526] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.100537] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.100543] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.100550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100562] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100573] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.100576] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.100582] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.100587] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.100590] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100601] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.100612] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.100615] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.100618] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.100643] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647100623), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.100652] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.100659] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.100671] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [DDL] execute ddl like stmt(ret=-4122, cost=2335, ddl_arg=NULL) [2024-09-11 06:20:47.203308] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54245536) [2024-09-11 06:20:47.204174] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5122, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.204390] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035647204060000, start_id=1726035647204060000, end_id=1726035647204060001) [2024-09-11 06:20:47.204886] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.204902] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.204908] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.205593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205643] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.205661] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.205669] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.205678] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205696] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645287009, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205712] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.205717] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.205725] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.205732] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035645278128, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.205769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.205812] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=43] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.205818] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.205850] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647205826), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.205860] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.205872] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.205878] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2573, ddl_arg=NULL) [2024-09-11 06:20:47.310985] INFO [RS] add_server (ob_root_service.cpp:7915) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54137760) [2024-09-11 06:20:47.311769] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5123, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.311961] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1726035647311576000, start_id=1726035647311576000, end_id=1726035647311576001) [2024-09-11 06:20:47.312399] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.312411] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.312419] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.313070] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313103] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313118] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.313131] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.313136] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.313143] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313153] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313164] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.313167] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.313170] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.313176] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.313179] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313191] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.313201] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.313204] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.313206] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.313227] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647313210), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.313235] INFO [RS] add_server (ob_root_service.cpp:7928) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.313242] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.313248] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55206][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2265, ddl_arg=NULL) [2024-09-11 06:20:47.415875] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=54032847) [2024-09-11 06:20:47.416631] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5124, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.416849] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035647416837000, start_id=1726035647416837000, end_id=1726035647416837001) [2024-09-11 06:20:47.417273] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.417286] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.417296] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.417971] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418000] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418013] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.418025] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.418032] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.418037] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418050] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418060] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.418063] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.418067] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.418071] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.418074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418084] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.418095] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.418098] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.418101] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.418124] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647418107), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.418131] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.418136] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.418140] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2266, ddl_arg=NULL) [2024-09-11 06:20:47.520640] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53928085) [2024-09-11 06:20:47.521402] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5125, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.521565] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035647521136000, start_id=1726035647521136000, end_id=1726035647521136001) [2024-09-11 06:20:47.522001] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.522018] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.522030] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.522675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522705] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522721] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.522732] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.522738] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.522744] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522756] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522772] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.522788] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.522796] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.522805] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.522809] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522821] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.522832] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.522835] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.522839] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.522867] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647522846), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.522876] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.522883] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.522889] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2251, ddl_arg=NULL) [2024-09-11 06:20:47.624963] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53823760) [2024-09-11 06:20:47.625771] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5126, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.625983] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=48] get number(ret=0, service_type_=0, range=1, base_id=1726035647625290000, start_id=1726035647625290000, end_id=1726035647625290001) [2024-09-11 06:20:47.626427] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:47.626438] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.626448] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.626456] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.627171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627201] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627217] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.627228] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.627234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.627240] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627257] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627270] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.627273] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.627278] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.627282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627293] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.627305] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.627307] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.627311] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.627335] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647627315), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.627345] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.627355] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.627360] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2400, ddl_arg=NULL) [2024-09-11 06:20:47.729707] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53719019) [2024-09-11 06:20:47.730478] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5127, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.730648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035647730438000, start_id=1726035647730438000, end_id=1726035647730438001) [2024-09-11 06:20:47.731017] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.731026] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.731032] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.731647] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731676] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731700] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.731712] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.731717] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.731725] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731737] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731748] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.731751] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.731754] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.731760] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.731763] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731782] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.731793] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.731795] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.731797] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.731818] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647731800), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.731825] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.731832] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.731838] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2132, ddl_arg=NULL) [2024-09-11 06:20:47.834631] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53614109) [2024-09-11 06:20:47.835631] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5128, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.835846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035647835778000, start_id=1726035647835778000, end_id=1726035647835778001) [2024-09-11 06:20:47.836255] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.836269] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.836280] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.837181] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=44] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837247] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:47.837261] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:47.837269] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:47.837278] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837296] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837311] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:47.837318] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.837326] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.837331] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:47.837338] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837356] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.837372] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:47.837376] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:47.837380] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:47.837407] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647837387), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.837418] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.837428] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:47.837436] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2806, ddl_arg=NULL) [2024-09-11 06:20:47.939250] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53509488) [2024-09-11 06:20:47.940002] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5129, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:47.940169] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:47.940565] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:47.940576] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:47.940584] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.941239] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.941274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:47.941323] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035647941306), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:47.941329] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:47.941336] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.044159] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53404566) [2024-09-11 06:20:48.045270] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1726035648044302000, start_id=1726035648044302000, end_id=1726035648044302001) [2024-09-11 06:20:48.045678] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.045690] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.045697] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.046353] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.046379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.046394] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.046405] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.046410] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.046414] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.046444] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648046428), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.046451] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.046458] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.148661] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53300065) [2024-09-11 06:20:48.149474] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5131, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.149642] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035648149588000, start_id=1726035648149588000, end_id=1726035648149588001) [2024-09-11 06:20:48.150063] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.150077] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.150087] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.150749] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.150801] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=51] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.150817] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.150827] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.150833] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.150838] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.150850] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:48.150885] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648150865), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.150893] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.150898] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.263902] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53184861) [2024-09-11 06:20:48.264717] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5132, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.264920] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035648264219000, start_id=1726035648264219000, end_id=1726035648264219001) [2024-09-11 06:20:48.265344] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.266034] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.266065] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.266080] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.266091] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.266097] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.266111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.266125] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.266136] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:48.266165] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648266143), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.266174] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.368694] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=53080032) [2024-09-11 06:20:48.369850] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5133, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.370016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035648369397000, start_id=1726035648369397000, end_id=1726035648369397001) [2024-09-11 06:20:48.371177] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.371207] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.371256] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648371236), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.371266] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.474419] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52974312) [2024-09-11 06:20:48.475171] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5134, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.475336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035648474696000, start_id=1726035648474696000, end_id=1726035648474696001) [2024-09-11 06:20:48.475769] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.475786] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.475793] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.476437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476464] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476479] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.476488] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.476494] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.476501] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476511] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476522] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:48.476524] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.476531] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:48.476537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.476558] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:48.476560] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.476562] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:48.476584] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648476568), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.476592] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.476598] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.476605] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2186, ddl_arg=NULL) [2024-09-11 06:20:48.579557] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52869172) [2024-09-11 06:20:48.580406] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5135, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.580586] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035648580034000, start_id=1726035648580034000, end_id=1726035648580034001) [2024-09-11 06:20:48.581025] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.581044] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.581113] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=68][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.581768] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=54] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581841] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.581853] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.581861] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.581870] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581883] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581899] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:48.581903] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.581907] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:48.581911] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:48.581916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581927] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.581938] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:48.581941] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.581943] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:48.581967] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648581947), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.581975] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.581980] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.581986] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2432, ddl_arg=NULL) [2024-09-11 06:20:48.683737] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52764992) [2024-09-11 06:20:48.684525] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5136, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.684693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035648684096000, start_id=1726035648684096000, end_id=1726035648684096001) [2024-09-11 06:20:48.685119] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:48.685131] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.685141] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.685151] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.685804] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685849] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.685859] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.685864] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.685872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:48.685902] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.685908] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.685913] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:48.685919] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685932] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.685942] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:48.685944] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.685946] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.685948] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:48.685970] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648685952), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.685976] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.685982] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.685985] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2251, ddl_arg=NULL) [2024-09-11 06:20:48.788403] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52660319) [2024-09-11 06:20:48.789175] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5137, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.789340] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035648789291000, start_id=1726035648789291000, end_id=1726035648789291001) [2024-09-11 06:20:48.789771] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.789806] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.789813] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.790441] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790471] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790487] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:48.790500] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:48.790506] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:48.790515] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790528] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790539] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:48.790542] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.790548] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.790551] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:48.790554] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790566] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:48.790574] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:48.790577] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:48.790579] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.790581] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:48.790603] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648790585), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.790610] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.790617] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:48.790621] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2220, ddl_arg=NULL) [2024-09-11 06:20:48.903405] INFO [RS] add_server (ob_root_service.cpp:7915) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52545329) [2024-09-11 06:20:48.904267] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5138, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:48.904491] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1726035648904079000, start_id=1726035648904079000, end_id=1726035648904079001) [2024-09-11 06:20:48.904948] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:48.904965] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:48.904974] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.905661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:48.905700] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.905709] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:48.905713] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:48.905734] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035648905718), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:48.905742] INFO [RS] add_server (ob_root_service.cpp:7928) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:48.905747] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.008315] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52440405) [2024-09-11 06:20:49.009099] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5139, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.009268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035649009107000, start_id=1726035649009107000, end_id=1726035649009107001) [2024-09-11 06:20:49.009673] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.009686] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.009695] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.010362] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010392] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647067188, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010407] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.010422] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.010428] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.010433] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010445] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010457] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.010460] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.010464] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.010468] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.010471] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010483] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.010501] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.010505] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.010508] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.010510] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.010534] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649010515), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.010542] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.010548] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.010552] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2239, ddl_arg=NULL) [2024-09-11 06:20:49.123575] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52325156) [2024-09-11 06:20:49.124343] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5140, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.124520] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649123699000, start_id=1726035649123699000, end_id=1726035649123699001) [2024-09-11 06:20:49.124946] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.124958] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.124968] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.125593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125621] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125637] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.125648] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.125654] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.125659] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125671] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125683] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.125686] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.125689] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.125694] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.125699] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125709] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647278506, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.125720] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.125722] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.125725] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.125727] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.125751] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649125733), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.125757] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.125763] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.125767] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2194, ddl_arg=NULL) [2024-09-11 06:20:49.228030] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52220712) [2024-09-11 06:20:49.228862] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:49.229437] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.229455] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.229467] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.230213] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035647287690, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.230239] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.230246] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.230275] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649230253), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.230286] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.230292] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.342976] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52105758) [2024-09-11 06:20:49.343749] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5142, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.343933] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649343736000, start_id=1726035649343736000, end_id=1726035649343736001) [2024-09-11 06:20:49.344316] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.344329] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.344341] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.344962] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.344991] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.345007] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.345017] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.345023] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.345028] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.345040] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.345052] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.345055] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.345059] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.345065] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.345068] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.345081] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.345090] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.345093] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.345096] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.345098] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.345118] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649345103), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.345126] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.345135] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.345139] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2165, ddl_arg=NULL) [2024-09-11 06:20:49.448533] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=52000199) [2024-09-11 06:20:49.449294] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5143, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.449448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649448894000, start_id=1726035649448894000, end_id=1726035649448894001) [2024-09-11 06:20:49.449950] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.450699] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450728] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450744] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.450756] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.450762] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.450767] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450807] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450817] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.450821] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.450825] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.450831] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.450835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450847] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.450859] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.450861] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.450864] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.450866] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.450888] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649450870), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.450896] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.450907] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2377, ddl_arg=NULL) [2024-09-11 06:20:49.563536] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51885198) [2024-09-11 06:20:49.564304] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5144, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.564469] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649563554000, start_id=1726035649563554000, end_id=1726035649563554001) [2024-09-11 06:20:49.564878] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.564889] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.564897] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.565546] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565584] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.565594] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.565599] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.565604] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565627] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.565629] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.565632] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.565637] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.565640] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.565663] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.565665] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.565668] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.565690] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649565671), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.565699] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.565705] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.565709] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2175, ddl_arg=NULL) [2024-09-11 06:20:49.668077] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51780644) [2024-09-11 06:20:49.668962] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5145, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.669185] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035649668742000, start_id=1726035649668742000, end_id=1726035649668742001) [2024-09-11 06:20:49.669609] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.669627] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.669636] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.670332] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670382] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.670394] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.670400] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.670406] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670418] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670429] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.670432] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.670438] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.670443] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.670447] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.670471] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.670473] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.670477] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.670480] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.670506] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649670485), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.670514] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.670522] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.670527] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2452, ddl_arg=NULL) [2024-09-11 06:20:49.772736] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51676006) [2024-09-11 06:20:49.773530] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5146, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.773697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649773115000, start_id=1726035649773115000, end_id=1726035649773115001) [2024-09-11 06:20:49.774127] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:49.774137] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.774145] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.774152] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.774817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.774848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.774863] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.774875] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.774881] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.774888] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.774900] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.774912] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:49.774915] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.774924] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.774949] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649774927), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.774958] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.774965] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.877553] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51571176) [2024-09-11 06:20:49.878307] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5147, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.878477] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649878333000, start_id=1726035649878333000, end_id=1726035649878333001) [2024-09-11 06:20:49.878944] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.878958] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.878969] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.879689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879720] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879735] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.879748] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.879754] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.879759] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879771] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879790] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.879792] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.879796] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.879802] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.879806] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.879830] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.879834] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.879836] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.879839] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.879864] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649879843), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.879873] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.879880] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.879887] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2336, ddl_arg=NULL) [2024-09-11 06:20:49.982131] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51466597) [2024-09-11 06:20:49.982902] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5148, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:49.983054] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035649982496000, start_id=1726035649982496000, end_id=1726035649982496001) [2024-09-11 06:20:49.983512] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:49.983524] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:49.983530] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.984155] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984185] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984200] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:49.984213] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:49.984219] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:49.984224] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984247] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984258] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:49.984261] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.984265] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.984268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:49.984273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:49.984295] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:49.984299] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:49.984303] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:49.984306] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:49.984327] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035649984310), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:49.984335] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:49.984343] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:49.984347] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2218, ddl_arg=NULL) [2024-09-11 06:20:50.086572] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51362151) [2024-09-11 06:20:50.087398] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5149, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.087576] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650086547000, start_id=1726035650086547000, end_id=1726035650086547001) [2024-09-11 06:20:50.087984] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.087997] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.088007] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.088664] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088693] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088709] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.088719] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.088725] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.088730] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088741] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088752] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.088757] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.088761] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.088765] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.088769] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088803] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.088813] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.088816] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.088819] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.088821] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.088844] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650088825), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.088852] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.088859] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.088866] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2296, ddl_arg=NULL) [2024-09-11 06:20:50.191353] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51257369) [2024-09-11 06:20:50.192204] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5150, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.192418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035650191685000, start_id=1726035650191685000, end_id=1726035650191685001) [2024-09-11 06:20:50.192910] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.192924] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.192931] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.193609] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193639] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193655] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.193665] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.193670] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.193677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193701] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.193704] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.193711] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.193715] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.193720] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193729] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.193738] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.193740] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.193743] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.193745] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.193769] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650193749), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.193793] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.193802] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.193808] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2457, ddl_arg=NULL) [2024-09-11 06:20:50.296046] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51152685) [2024-09-11 06:20:50.296875] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5151, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.297112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650297001000, start_id=1726035650297001000, end_id=1726035650297001001) [2024-09-11 06:20:50.297587] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.297600] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.297608] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.298310] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298340] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298358] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.298372] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.298379] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.298386] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298398] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298409] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.298411] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.298415] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.298420] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.298423] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298434] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.298445] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.298448] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.298451] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.298453] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.298480] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650298457), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.298489] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.298495] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.298499] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2455, ddl_arg=NULL) [2024-09-11 06:20:50.400522] INFO [RS] add_server (ob_root_service.cpp:7915) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=51048201) [2024-09-11 06:20:50.401383] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5152, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.401563] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035650401114000, start_id=1726035650401114000, end_id=1726035650401114001) [2024-09-11 06:20:50.401969] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.401993] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.402001] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.402652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.402678] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.402699] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.402711] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:50.402720] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.402726] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.402751] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650402730), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.402759] INFO [RS] add_server (ob_root_service.cpp:7928) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.402767] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.504957] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50943770) [2024-09-11 06:20:50.505696] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5153, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.505882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650505110000, start_id=1726035650505110000, end_id=1726035650505110001) [2024-09-11 06:20:50.506321] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.506336] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:50.507063] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507095] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507111] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.507122] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.507129] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.507134] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507146] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507158] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.507161] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.507164] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.507168] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.507171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507183] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.507195] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.507197] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.507200] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.507203] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.507225] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650507206), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.507233] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.507243] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2288, ddl_arg=NULL) [2024-09-11 06:20:50.642134] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50806591) [2024-09-11 06:20:50.642958] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5154, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.643129] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035650643116000, start_id=1726035650643116000, end_id=1726035650643116001) [2024-09-11 06:20:50.643538] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.643551] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.643560] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.644218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.644249] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.644264] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.644274] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.644280] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.644287] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.644299] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.644311] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.644313] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.644316] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.644322] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:50.644331] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.644363] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650644338), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.644371] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.644378] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.746614] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50702118) [2024-09-11 06:20:50.747418] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5155, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.747618] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650747114000, start_id=1726035650747114000, end_id=1726035650747114001) [2024-09-11 06:20:50.748013] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.748026] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.748035] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.748675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748704] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748720] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.748730] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.748737] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.748743] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748766] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.748769] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.748783] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.748786] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.748790] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748803] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.748815] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.748818] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.748820] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.748822] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.748843] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650748825), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.748854] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.748860] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.748864] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2252, ddl_arg=NULL) [2024-09-11 06:20:50.851553] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50597177) [2024-09-11 06:20:50.852341] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5156, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.852499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650852273000, start_id=1726035650852273000, end_id=1726035650852273001) [2024-09-11 06:20:50.852903] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:50.852914] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.852924] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.852929] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.853611] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853639] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853656] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.853666] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.853673] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.853677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853701] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.853704] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.853710] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.853713] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.853718] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853732] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.853748] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.853752] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.853754] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.853757] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.853789] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650853763), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.853798] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.853804] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.853808] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2257, ddl_arg=NULL) [2024-09-11 06:20:50.956001] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50492737) [2024-09-11 06:20:50.956792] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5157, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:50.956979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035650956255000, start_id=1726035650956255000, end_id=1726035650956255001) [2024-09-11 06:20:50.957425] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:50.957437] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:50.957446] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.958105] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958136] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958152] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:50.958161] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:50.958167] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:50.958174] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958185] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958195] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:50.958199] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.958202] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.958206] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:50.958208] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958219] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:50.958229] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:50.958232] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:50.958234] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:50.958237] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:50.958263] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035650958243), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:50.958272] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:50.958277] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:50.958282] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2283, ddl_arg=NULL) [2024-09-11 06:20:51.060568] INFO [RS] add_server (ob_root_service.cpp:7915) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50388154) [2024-09-11 06:20:51.061365] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5158, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.061532] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035651061436000, start_id=1726035651061436000, end_id=1726035651061436001) [2024-09-11 06:20:51.061943] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.061957] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.061967] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.062630] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649067518, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062680] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.062693] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.062700] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.062705] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062718] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062736] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.062739] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.062742] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.062745] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.062748] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062758] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.062770] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.062800] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.062803] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.062806] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.062828] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651062809), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.062836] INFO [RS] add_server (ob_root_service.cpp:7928) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.062843] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.062849] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55205][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2283, ddl_arg=NULL) [2024-09-11 06:20:51.165107] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50283629) [2024-09-11 06:20:51.165874] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5159, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.166037] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035651165581000, start_id=1726035651165581000, end_id=1726035651165581001) [2024-09-11 06:20:51.166439] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.166451] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.166459] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.167106] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167136] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167149] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.167158] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.167164] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.167169] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167179] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167190] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.167193] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.167197] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.167200] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.167203] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167213] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.167224] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.167226] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.167228] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.167230] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.167252] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651167234), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.167260] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.167264] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.167268] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2163, ddl_arg=NULL) [2024-09-11 06:20:51.269795] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50178935) [2024-09-11 06:20:51.270647] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5160, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.270862] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035651269835000, start_id=1726035651269835000, end_id=1726035651269835001) [2024-09-11 06:20:51.271278] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.271292] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.271301] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.271975] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272004] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272019] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.272031] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.272036] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.272043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272056] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649288286, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272068] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.272071] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.272076] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.272080] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.272085] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035649278884, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.272109] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.272112] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.272115] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.272117] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.272138] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651272120), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.272149] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.272160] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.272165] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2372, ddl_arg=NULL) [2024-09-11 06:20:51.374808] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=50073926) [2024-09-11 06:20:51.375562] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5161, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.375714] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035651375056000, start_id=1726035651375056000, end_id=1726035651375056001) [2024-09-11 06:20:51.376144] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.376157] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.376163] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.376810] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376846] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376859] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.376868] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.376873] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.376883] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376903] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376916] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.376920] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.376923] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.376927] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.376931] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376943] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.376954] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.376956] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.376958] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.376960] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.376985] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651376964), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.376993] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.377000] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.377006] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2200, ddl_arg=NULL) [2024-09-11 06:20:51.482464] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49966265) [2024-09-11 06:20:51.483296] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5162, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.483478] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035651482503000, start_id=1726035651482503000, end_id=1726035651482503001) [2024-09-11 06:20:51.483931] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.483945] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.483953] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.484603] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484651] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.484664] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.484670] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.484675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484685] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484697] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.484700] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.484706] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.484709] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.484712] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484723] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.484732] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.484735] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:51.484737] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.484759] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651484741), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.484767] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.484784] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.484788] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2326, ddl_arg=NULL) [2024-09-11 06:20:51.587164] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49861563) [2024-09-11 06:20:51.588199] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5163, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.588407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035651587853000, start_id=1726035651587853000, end_id=1726035651587853001) [2024-09-11 06:20:51.588885] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.588900] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:51.589808] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589844] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589867] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.589881] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.589888] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.589897] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589935] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.589939] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.589945] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.589949] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.589957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.589990] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.589997] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.590000] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.590006] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.590035] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651590015), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.590045] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.590056] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2896, ddl_arg=NULL) [2024-09-11 06:20:51.691619] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49757109) [2024-09-11 06:20:51.692567] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5164, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.692750] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035651692017000, start_id=1726035651692017000, end_id=1726035651692017001) [2024-09-11 06:20:51.693200] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.693215] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.693232] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.693943] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.693970] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.693985] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.693997] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.694003] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.694009] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.694021] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.694032] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.694035] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.694038] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.694041] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.694047] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.694058] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.694069] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.694072] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.694074] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.694076] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.694100] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651694082), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.694107] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.694115] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.694122] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2505, ddl_arg=NULL) [2024-09-11 06:20:51.796476] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49652250) [2024-09-11 06:20:51.797269] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5165, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.797439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035651797424000, start_id=1726035651797424000, end_id=1726035651797424001) [2024-09-11 06:20:51.797871] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.797884] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.797895] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.798527] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.798553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.798570] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.798579] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.798585] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.798596] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:51.798610] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.798617] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.798637] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651798620), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.798644] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.798651] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.901369] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49547354) [2024-09-11 06:20:51.902153] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5166, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:51.902331] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035651901690000, start_id=1726035651901690000, end_id=1726035651901690001) [2024-09-11 06:20:51.902763] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:51.902810] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=46][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:51.902820] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:51.902827] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.903456] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903484] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903501] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:51.903511] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:20:51.903517] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:51.903524] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903534] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903545] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:51.903547] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.903551] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.903556] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:51.903559] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:51.903581] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:51.903583] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:51.903585] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:51.903588] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:51.903610] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035651903591), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:51.903618] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:51.903625] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:51.903630] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2263, ddl_arg=NULL) [2024-09-11 06:20:52.006087] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49442652) [2024-09-11 06:20:52.006838] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5167, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.007038] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035652006895000, start_id=1726035652006895000, end_id=1726035652006895001) [2024-09-11 06:20:52.007473] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.007484] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.007492] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.008190] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.008218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.008231] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.008243] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.008249] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.008254] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.008265] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.008276] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:52.008279] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.008281] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.008286] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.008309] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652008291), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.008314] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.008321] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.111555] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49337173) [2024-09-11 06:20:52.112331] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5168, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.112549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035652112183000, start_id=1726035652112183000, end_id=1726035652112183001) [2024-09-11 06:20:52.113042] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.113057] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.113064] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.113691] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.113719] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.113734] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.113750] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.113757] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.113805] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652113761), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.113814] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.113819] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.216730] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49232016) [2024-09-11 06:20:52.217813] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5169, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.218043] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035652217315000, start_id=1726035652217315000, end_id=1726035652217315001) [2024-09-11 06:20:52.218513] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.218530] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.218541] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.219404] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.219441] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.219465] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.219481] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.219489] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.219499] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.219521] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.219528] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.219557] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652219536), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.219565] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.219576] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.321942] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49126800) [2024-09-11 06:20:52.322738] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5170, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.322933] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035652322557000, start_id=1726035652322557000, end_id=1726035652322557001) [2024-09-11 06:20:52.323375] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.323389] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.323398] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.324061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.324089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.324105] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.324115] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.324121] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.324128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.324139] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:52.324152] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.324158] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:52.324162] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.324185] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652324166), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.324203] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.324208] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.427124] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49021602) [2024-09-11 06:20:52.428265] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5171, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.428667] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1726035652427631000, start_id=1726035652427631000, end_id=1726035652427631001) [2024-09-11 06:20:52.429169] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.429187] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.429196] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.430042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.430083] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.430105] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.430122] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.430131] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.430141] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.430178] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=36] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:52.430192] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.430200] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.430231] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652430207), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.430242] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.430253] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.532289] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:52.533084] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5172, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.533248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035652532743000, start_id=1726035652532743000, end_id=1726035652532743001) [2024-09-11 06:20:52.533655] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.533669] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.533677] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.534411] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.534421] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.534443] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652534425), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.534450] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.534456] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.637675] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48811058) [2024-09-11 06:20:52.638783] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5173, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.638992] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1726035652637945000, start_id=1726035652637945000, end_id=1726035652637945001) [2024-09-11 06:20:52.639421] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.639439] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.639450] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.640385] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640416] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640432] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.640443] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.640448] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.640455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640470] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640481] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:52.640484] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.640487] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.640490] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:52.640494] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.640516] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:52.640519] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.640523] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.640525] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:52.640550] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652640530), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.640561] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.640572] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.640580] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2908, ddl_arg=NULL) [2024-09-11 06:20:52.753467] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48695266) [2024-09-11 06:20:52.754227] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5174, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.754419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035652753677000, start_id=1726035652753677000, end_id=1726035652753677001) [2024-09-11 06:20:52.755586] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755616] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755634] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.755644] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.755648] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.755653] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755664] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755675] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:52.755678] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.755681] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.755687] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:52.755690] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755700] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.755717] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:52.755721] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.755724] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.755729] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:52.755749] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652755732), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.755756] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.755765] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2300, ddl_arg=NULL) [2024-09-11 06:20:52.858738] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48589981) [2024-09-11 06:20:52.859510] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5175, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:52.859678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035652858958000, start_id=1726035652858958000, end_id=1726035652858958001) [2024-09-11 06:20:52.860094] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.860107] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.860115] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.860767] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.860823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=56] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.860840] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:52.860850] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:52.860857] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:52.860865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.860876] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.860886] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:52.860889] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:52.860895] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.860899] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:52.860901] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.860913] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:52.860925] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.860948] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652860928), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.860956] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.860962] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:52.964044] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:52.965394] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:52.965406] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:52.965417] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:52.965424] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.966107] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.966135] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651067860, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:52.966159] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.966168] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:52.966189] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035652966172), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:52.966197] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:52.966205] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.069053] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48379669) [2024-09-11 06:20:53.070094] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5177, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.070319] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1726035653069278000, start_id=1726035653069278000, end_id=1726035653069278001) [2024-09-11 06:20:53.070841] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.070858] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.070870] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.071790] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.071826] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.071858] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.071869] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.071895] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653071874), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.071913] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.071925] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.174161] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48274574) [2024-09-11 06:20:53.174937] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5178, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.175111] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035653174290000, start_id=1726035653174290000, end_id=1726035653174290001) [2024-09-11 06:20:53.175519] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.175534] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.175544] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.176170] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.176199] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.176215] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.176228] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.176234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.176239] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.176250] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.176261] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:53.176265] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.176271] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.176275] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:53.176278] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651279292, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.176289] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:53.176302] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.176328] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653176306), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.176337] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.176342] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.279423] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48169304) [2024-09-11 06:20:53.280232] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5179, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.280420] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035653280419000, start_id=1726035653280419000, end_id=1726035653280419001) [2024-09-11 06:20:53.280853] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.280868] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.280878] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.281541] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281574] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281589] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.281602] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.281609] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.281615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281627] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035651288899, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281639] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:53.281642] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.281646] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.281650] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:53.281655] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281670] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.281687] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:53.281690] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.281693] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.281697] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:53.281720] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653281701), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.281730] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.281737] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.281744] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2323, ddl_arg=NULL) [2024-09-11 06:20:53.384568] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=48064161) [2024-09-11 06:20:53.385365] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5180, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.385527] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035653385468000, start_id=1726035653385468000, end_id=1726035653385468001) [2024-09-11 06:20:53.385960] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.385971] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.385979] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.386631] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386663] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386678] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.386690] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.386695] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.386700] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386711] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386721] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:53.386724] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.386727] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.386732] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:53.386735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386747] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.386758] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:53.386761] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.386763] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.386766] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:53.386798] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653386769), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.386806] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.386811] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.386817] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2251, ddl_arg=NULL) [2024-09-11 06:20:53.489496] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47959227) [2024-09-11 06:20:53.490245] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5181, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.490408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035653489467000, start_id=1726035653489467000, end_id=1726035653489467001) [2024-09-11 06:20:53.490872] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.490888] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.490896] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.491558] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491585] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491600] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.491612] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.491620] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.491627] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491639] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491650] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:53.491653] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.491659] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.491662] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:53.491665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491674] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.491684] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:53.491687] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.491689] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.491691] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:53.491710] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653491694), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.491717] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.491724] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.491728] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2234, ddl_arg=NULL) [2024-09-11 06:20:53.594495] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47854228) [2024-09-11 06:20:53.595267] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5182, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.595437] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035653594751000, start_id=1726035653594751000, end_id=1726035653594751001) [2024-09-11 06:20:53.595503] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=38, avg_log_batch_cnt=1, total_group_log_size=29713, avg_group_log_size=781) [2024-09-11 06:20:53.595516] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=29524) [2024-09-11 06:20:53.595886] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.595901] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.595911] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.596596] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.596623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.596638] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.596648] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.596654] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.596659] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:53.596671] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.596676] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.596698] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653596680), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.596707] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.596712] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.699055] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47749674) [2024-09-11 06:20:53.699793] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5183, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.699969] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035653699057000, start_id=1726035653699057000, end_id=1726035653699057001) [2024-09-11 06:20:53.700382] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.700394] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.700403] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.701061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701104] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.701114] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.701121] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.701126] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701137] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701149] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:53.701151] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.701155] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.701158] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:53.701163] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701175] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.701185] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:53.701188] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:53.701192] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.701199] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:53.701220] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653701203), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.701228] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.701234] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:53.701241] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2187, ddl_arg=NULL) [2024-09-11 06:20:53.803581] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47645155) [2024-09-11 06:20:53.804362] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5184, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.804511] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035653804274000, start_id=1726035653804274000, end_id=1726035653804274001) [2024-09-11 06:20:53.804932] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.804944] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:53.805608] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.805635] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.805650] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:53.805662] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:53.805666] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:53.805674] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.805680] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.805700] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653805683), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.805707] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.908038] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47540688) [2024-09-11 06:20:53.908945] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5185, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:53.909119] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035653908364000, start_id=1726035653908364000, end_id=1726035653908364001) [2024-09-11 06:20:53.909532] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:53.909545] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:53.909554] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.910210] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:53.910239] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:53.910265] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.910278] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:53.910299] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035653910284), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:53.910308] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:53.910315] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.023121] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47425612) [2024-09-11 06:20:54.023900] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5186, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.024079] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035654023963000, start_id=1726035654023963000, end_id=1726035654023963001) [2024-09-11 06:20:54.024491] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:54.024502] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.024513] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.024520] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.025205] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025249] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.025260] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.025266] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.025273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025296] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.025299] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.025304] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.025308] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.025312] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025322] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.025331] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.025334] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.025336] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.025339] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.025364] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654025342), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.025372] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.025380] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.025386] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2267, ddl_arg=NULL) [2024-09-11 06:20:54.128470] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47320249) [2024-09-11 06:20:54.129274] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5187, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.129454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035654129185000, start_id=1726035654129185000, end_id=1726035654129185001) [2024-09-11 06:20:54.129880] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.129891] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.129897] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.130518] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130547] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130563] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.130573] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.130579] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.130584] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130595] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130605] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.130608] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.130611] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.130616] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.130619] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130628] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.130636] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.130639] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.130641] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.130643] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.130664] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654130646), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.130671] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.130678] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.130682] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:20:54.243158] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47205568) [2024-09-11 06:20:54.243947] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5188, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.244129] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035654243944000, start_id=1726035654243944000, end_id=1726035654243944001) [2024-09-11 06:20:54.244546] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.244560] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.244568] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.245254] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245299] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.245314] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.245323] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.245331] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245362] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.245365] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.245368] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.245374] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.245378] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245393] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.245404] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.245407] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.245410] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.245412] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.245431] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654245416), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.245438] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.245447] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.245452] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2296, ddl_arg=NULL) [2024-09-11 06:20:54.347467] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=47101256) [2024-09-11 06:20:54.348250] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5189, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.348422] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035654348182000, start_id=1726035654348182000, end_id=1726035654348182001) [2024-09-11 06:20:54.348847] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.348864] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.348870] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.349513] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349541] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349556] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.349569] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.349575] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.349580] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349594] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349605] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.349608] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.349611] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.349617] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.349620] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.349638] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.349641] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.349643] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.349645] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.349664] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654349648), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.349671] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.349676] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.349680] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:20:54.452162] INFO [RS] add_server (ob_root_service.cpp:7915) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46996567) [2024-09-11 06:20:54.452962] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5190, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.453143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035654452284000, start_id=1726035654452284000, end_id=1726035654452284001) [2024-09-11 06:20:54.453510] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.453521] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.453527] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.454306] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=46] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454386] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.454402] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.454413] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.454423] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454443] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454459] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.454468] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.454474] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.454479] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.454481] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454501] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.454516] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.454520] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.454524] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.454533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.454578] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654454546), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.454586] INFO [RS] add_server (ob_root_service.cpp:7928) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.454591] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.454594] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2433, ddl_arg=NULL) [2024-09-11 06:20:54.556689] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46892037) [2024-09-11 06:20:54.557417] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5191, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.557589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035654557518000, start_id=1726035654557518000, end_id=1726035654557518001) [2024-09-11 06:20:54.558029] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.558041] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.558049] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.558680] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558707] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558722] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.558730] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.558736] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.558743] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558764] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.558766] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.558770] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.558781] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.558784] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558794] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.558805] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.558808] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.558810] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.558813] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.558834] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654558819), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.558842] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.558848] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.558853] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2165, ddl_arg=NULL) [2024-09-11 06:20:54.661484] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46787243) [2024-09-11 06:20:54.662319] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5192, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.662501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035654661782000, start_id=1726035654661782000, end_id=1726035654661782001) [2024-09-11 06:20:54.662560] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=36, avg_log_batch_cnt=1, total_group_log_size=28382, avg_group_log_size=788) [2024-09-11 06:20:54.662570] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28382) [2024-09-11 06:20:54.662929] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.662941] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.662949] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.663615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663653] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663669] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.663679] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.663685] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.663690] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663702] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663714] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.663717] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.663722] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.663728] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.663733] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663744] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.663754] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.663757] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.663760] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.663763] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.663814] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654663767), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.663823] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.663829] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.663837] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2356, ddl_arg=NULL) [2024-09-11 06:20:54.766286] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46682449) [2024-09-11 06:20:54.767075] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5193, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.767245] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035654766845000, start_id=1726035654766845000, end_id=1726035654766845001) [2024-09-11 06:20:54.767693] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.767710] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:54.767720] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.768437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768466] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768482] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.768493] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.768500] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.768505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768516] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768529] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.768531] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.768535] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.768538] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.768542] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768554] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.768565] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.768568] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.768572] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.768574] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.768597] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654768578), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.768603] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.768612] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:54.768616] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2332, ddl_arg=NULL) [2024-09-11 06:20:54.870843] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46577899) [2024-09-11 06:20:54.871626] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5194, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.871794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035654871198000, start_id=1726035654871198000, end_id=1726035654871198001) [2024-09-11 06:20:54.872228] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:54.872241] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:54.872964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.872993] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.873009] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.873019] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.873025] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.873032] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.873044] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.873056] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.873059] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.873062] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.873066] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.873069] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.873081] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.873091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.873093] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.873095] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.873098] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.873123] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654873104), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.873130] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.975380] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46473353) [2024-09-11 06:20:54.976159] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5195, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:54.976321] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035654975348000, start_id=1726035654975348000, end_id=1726035654975348001) [2024-09-11 06:20:54.976763] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:54.977427] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653068226, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977471] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:54.977490] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] added async task to update rslist(force_update=false) [2024-09-11 06:20:54.977496] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:54.977502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977515] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977525] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:54.977533] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.977544] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.977548] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:54.977552] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977564] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:54.977576] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:54.977579] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:54.977581] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:54.977583] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:54.977603] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035654977587), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:54.977618] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:54.977628] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2260, ddl_arg=NULL) [2024-09-11 06:20:55.080118] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46368611) [2024-09-11 06:20:55.080923] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5196, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.081104] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035655080472000, start_id=1726035655080472000, end_id=1726035655080472001) [2024-09-11 06:20:55.081535] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:55.081546] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.081553] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.081560] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.082259] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082286] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082301] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.082313] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.082318] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.082325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082336] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082347] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.082349] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.082352] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.082357] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.082360] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082371] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.082381] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.082383] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.082385] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.082387] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.082411] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655082394), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.082419] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.082426] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.082432] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2316, ddl_arg=NULL) [2024-09-11 06:20:55.184593] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46264134) [2024-09-11 06:20:55.185374] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5197, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.185547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035655184652000, start_id=1726035655184652000, end_id=1726035655184652001) [2024-09-11 06:20:55.185997] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.186012] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.186021] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.186666] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186696] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186712] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.186722] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.186725] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.186732] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186746] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653289437, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186757] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.186760] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.186764] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.186767] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.186771] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186789] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035653279704, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.186801] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.186804] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.186807] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.186809] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.186834] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655186813), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.186843] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.186848] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.186853] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2262, ddl_arg=NULL) [2024-09-11 06:20:55.289246] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46159480) [2024-09-11 06:20:55.290048] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5198, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.290210] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035655289891000, start_id=1726035655289891000, end_id=1726035655289891001) [2024-09-11 06:20:55.290640] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.290652] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.290661] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.291326] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291354] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291369] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.291382] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.291387] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.291392] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291403] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291415] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.291418] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.291424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.291428] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.291432] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291444] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.291454] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.291456] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.291459] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.291462] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.291484] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655291466), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.291492] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.291498] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.291504] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:20:55.393955] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=46054771) [2024-09-11 06:20:55.394843] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5199, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.395039] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035655394040000, start_id=1726035655394040000, end_id=1726035655394040001) [2024-09-11 06:20:55.395488] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.395503] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.395514] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.396218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396249] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396263] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.396273] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.396277] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.396283] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396295] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396307] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.396310] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.396314] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.396318] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.396326] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396341] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.396352] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.396356] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.396359] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.396361] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.396385] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655396364), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.396395] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.396402] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.396408] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2455, ddl_arg=NULL) [2024-09-11 06:20:55.498618] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45950117) [2024-09-11 06:20:55.499335] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5200, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.499510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035655499156000, start_id=1726035655499156000, end_id=1726035655499156001) [2024-09-11 06:20:55.499920] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.499933] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.499942] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.500579] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500629] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.500641] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.500647] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.500654] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500684] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.500687] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.500691] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.500694] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.500698] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500709] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.500720] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.500723] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.500726] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.500728] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.500750] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655500732), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.500761] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.500770] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.500808] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] [DDL] execute ddl like stmt(ret=-4122, cost=2192, ddl_arg=NULL) [2024-09-11 06:20:55.613861] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45834875) [2024-09-11 06:20:55.614690] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5201, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.614915] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035655613889000, start_id=1726035655613889000, end_id=1726035655613889001) [2024-09-11 06:20:55.615333] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.615347] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.615355] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.616019] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.616047] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.616063] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.616075] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.616081] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.616088] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.616100] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.616109] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.616112] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:55.616118] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.616123] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.616150] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655616129), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.616157] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.616164] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.616171] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2313, ddl_arg=NULL) [2024-09-11 06:20:55.718554] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45730169) [2024-09-11 06:20:55.719395] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5202, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.719570] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035655718789000, start_id=1726035655718789000, end_id=1726035655718789001) [2024-09-11 06:20:55.720000] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.720014] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.720023] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.720689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.720719] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.720745] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.720752] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.720780] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655720756), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.720794] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.720799] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.833754] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45614967) [2024-09-11 06:20:55.834517] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5203, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.834686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035655834531000, start_id=1726035655834531000, end_id=1726035655834531001) [2024-09-11 06:20:55.835085] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.835098] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.835105] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.835762] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835828] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=65] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835843] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.835852] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.835858] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.835865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835876] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835886] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.835889] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.835896] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.835900] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.835904] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.835928] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.835930] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.835933] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.835938] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.835960] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655835941), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.835968] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.835975] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.835978] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:55.938350] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45510372) [2024-09-11 06:20:55.939131] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5204, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:55.939323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1726035655938741000, start_id=1726035655938741000, end_id=1726035655938741001) [2024-09-11 06:20:55.939799] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:55.939814] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:55.939823] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.940454] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940500] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:55.940514] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:55.940521] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:55.940526] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940539] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940551] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:55.940554] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.940559] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.940565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:55.940569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:55.940590] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:55.940594] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:55.940599] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:55.940602] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:55.940624] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035655940605), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:55.940632] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:55.940640] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:55.940653] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] [DDL] execute ddl like stmt(ret=-4122, cost=2306, ddl_arg=NULL) [2024-09-11 06:20:56.053301] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45395430) [2024-09-11 06:20:56.054107] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5205, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.054268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035656053576000, start_id=1726035656053576000, end_id=1726035656053576001) [2024-09-11 06:20:56.054593] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.054606] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.055245] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055276] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055297] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.055309] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.055315] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.055321] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055332] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055342] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.055345] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.055350] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.055357] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.055360] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055372] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.055381] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.055384] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.055386] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.055389] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.055415] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656055392), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.055423] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.055431] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2132, ddl_arg=NULL) [2024-09-11 06:20:56.157887] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45290838) [2024-09-11 06:20:56.158643] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5206, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.158850] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035656158756000, start_id=1726035656158756000, end_id=1726035656158756001) [2024-09-11 06:20:56.159270] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:56.159280] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.159287] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.159292] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.159923] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.159953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.159974] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.159988] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.159993] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.160000] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.160011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.160029] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.160032] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.160035] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.160038] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.160044] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.160053] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.160064] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.160067] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.160069] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.160072] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.160091] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656160075), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.160098] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.160106] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.160111] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2227, ddl_arg=NULL) [2024-09-11 06:20:56.262558] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45186168) [2024-09-11 06:20:56.263275] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5207, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.263427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035656262903000, start_id=1726035656262903000, end_id=1726035656262903001) [2024-09-11 06:20:56.263815] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.263829] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.263838] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.264474] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264518] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.264529] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.264536] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.264541] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264551] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264563] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.264566] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.264573] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.264577] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.264580] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264592] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.264604] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.264606] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.264609] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.264612] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.264633] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656264615), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.264641] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.264647] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.264650] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2094, ddl_arg=NULL) [2024-09-11 06:20:56.367112] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=45081619) [2024-09-11 06:20:56.367969] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5208, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.368146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035656367911000, start_id=1726035656367911000, end_id=1726035656367911001) [2024-09-11 06:20:56.368533] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.368547] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.368556] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.369242] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369272] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369287] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.369298] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.369305] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.369311] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369323] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369334] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.369336] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.369342] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.369345] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.369350] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369362] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.369372] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.369374] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.369376] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.369378] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.369401] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656369382), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.369410] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.369417] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.369422] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2312, ddl_arg=NULL) [2024-09-11 06:20:56.472047] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44976682) [2024-09-11 06:20:56.472884] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5209, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.473062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035656473025000, start_id=1726035656473025000, end_id=1726035656473025001) [2024-09-11 06:20:56.473484] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.473503] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.473512] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.474198] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474230] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474248] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.474261] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.474268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.474273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474297] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.474300] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.474304] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.474309] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.474315] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474328] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.474337] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.474342] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.474345] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.474348] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.474373] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656474355), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.474381] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.474390] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.474394] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2350, ddl_arg=NULL) [2024-09-11 06:20:56.576487] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44872239) [2024-09-11 06:20:56.577296] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5210, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.577456] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035656577217000, start_id=1726035656577217000, end_id=1726035656577217001) [2024-09-11 06:20:56.577893] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.577906] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.577915] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.578597] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578632] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578649] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.578660] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.578667] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.578672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578683] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578696] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.578699] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.578704] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.578710] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.578713] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578726] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.578736] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.578743] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.578746] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.578749] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.578771] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656578752), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.578790] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.578797] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.578806] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2321, ddl_arg=NULL) [2024-09-11 06:20:56.679860] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44768867) [2024-09-11 06:20:56.681041] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5211, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.681274] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035656681173000, start_id=1726035656681173000, end_id=1726035656681173001) [2024-09-11 06:20:56.681711] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.681727] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.681739] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.682693] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682731] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682753] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.682767] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.682788] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.682798] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682835] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.682840] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.682847] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.682853] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.682859] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682878] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.682897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.682902] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.682906] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.682910] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.682940] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656682915), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.682952] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.682962] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.682969] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=3110, ddl_arg=NULL) [2024-09-11 06:20:56.784346] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44664396) [2024-09-11 06:20:56.785168] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5212, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.785340] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035656785222000, start_id=1726035656785222000, end_id=1726035656785222001) [2024-09-11 06:20:56.785764] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.785816] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=48][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.785826] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.786448] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786482] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786499] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.786511] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.786517] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.786524] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786553] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.786557] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.786564] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.786569] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.786578] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786591] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.786603] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.786606] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.786608] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.786611] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.786633] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656786615), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.786641] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.786647] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.786651] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2307, ddl_arg=NULL) [2024-09-11 06:20:56.888827] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44559917) [2024-09-11 06:20:56.889928] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5213, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:56.890175] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035656889485000, start_id=1726035656889485000, end_id=1726035656889485001) [2024-09-11 06:20:56.890614] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:56.890643] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:56.890652] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.891532] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=36] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891591] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:56.891607] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:56.891615] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:56.891624] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891644] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891663] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:56.891667] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.891673] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.891680] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:56.891694] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891712] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:56.891731] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:56.891738] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:56.891742] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:56.891748] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:56.891822] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035656891757), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:56.891833] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:56.891843] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:56.891851] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=3026, ddl_arg=NULL) [2024-09-11 06:20:57.003840] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44444939) [2024-09-11 06:20:57.004695] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5214, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.004942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035657004363000, start_id=1726035657004363000, end_id=1726035657004363001) [2024-09-11 06:20:57.005401] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.005415] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.005424] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.006133] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006159] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655068596, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006176] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.006186] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.006192] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.006199] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006212] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006223] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.006226] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.006231] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.006234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.006239] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.006261] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.006264] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.006266] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.006268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.006297] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657006274), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.006304] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.006312] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.006318] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2479, ddl_arg=NULL) [2024-09-11 06:20:57.108354] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44340373) [2024-09-11 06:20:57.109109] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5215, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.109305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657108522000, start_id=1726035657108522000, end_id=1726035657108522001) [2024-09-11 06:20:57.109728] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.109741] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.109750] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:57.110450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110506] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.110518] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.110524] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.110531] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110544] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110556] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.110559] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.110562] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.110565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.110569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.110591] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.110594] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.110597] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.110600] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.110621] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657110604), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.110630] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.110639] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2286, ddl_arg=NULL) [2024-09-11 06:20:57.223173] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44225552) [2024-09-11 06:20:57.223971] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5216, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.224127] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657223346000, start_id=1726035657223346000, end_id=1726035657223346001) [2024-09-11 06:20:57.224551] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:57.224562] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.224570] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.224576] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.225225] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225254] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225271] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.225281] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.225287] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.225292] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225303] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655289975, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225314] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.225317] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.225322] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.225325] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.225328] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225338] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035655280099, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.225349] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.225351] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.225354] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.225356] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.225376] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657225359), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.225385] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.225391] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.225395] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2223, ddl_arg=NULL) [2024-09-11 06:20:57.327729] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44121032) [2024-09-11 06:20:57.328524] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5217, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.328704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035657328452000, start_id=1726035657328452000, end_id=1726035657328452001) [2024-09-11 06:20:57.329112] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.329124] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.329132] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.329807] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329834] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329849] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.329859] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.329865] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.329871] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329882] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329893] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.329895] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.329899] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.329902] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.329905] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329914] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.329925] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.329928] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.329930] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.329932] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.329953] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657329936), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.329960] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.329965] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.329971] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2244, ddl_arg=NULL) [2024-09-11 06:20:57.432291] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=44016437) [2024-09-11 06:20:57.433065] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:57.433620] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.433633] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.433642] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.434334] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.434348] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.434354] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.434376] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657434357), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.434385] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.434394] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.536811] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43911921) [2024-09-11 06:20:57.537536] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5219, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.537706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035657537425000, start_id=1726035657537425000, end_id=1726035657537425001) [2024-09-11 06:20:57.538108] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.538119] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.538127] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.538746] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538812] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.538824] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.538830] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.538835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538853] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538865] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.538867] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.538871] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.538874] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.538877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538887] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.538897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.538899] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.538902] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.538904] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.538923] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657538907), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.538931] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.538938] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.538942] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2133, ddl_arg=NULL) [2024-09-11 06:20:57.641584] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43807142) [2024-09-11 06:20:57.642420] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5220, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.642584] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657642541000, start_id=1726035657642541000, end_id=1726035657642541001) [2024-09-11 06:20:57.643005] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.643023] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.643030] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.643682] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643716] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643730] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.643740] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.643746] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.643752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643763] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643781] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.643784] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.643787] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.643791] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.643795] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643805] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.643816] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.643819] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.643822] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.643824] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.643848] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657643827), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.643858] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.643863] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.643868] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2286, ddl_arg=NULL) [2024-09-11 06:20:57.745951] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43702779) [2024-09-11 06:20:57.746735] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5221, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.746955] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657746558000, start_id=1726035657746558000, end_id=1726035657746558001) [2024-09-11 06:20:57.747337] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.747350] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.747358] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.748022] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748056] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748073] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.748084] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.748090] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.748097] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748108] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748118] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.748121] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.748127] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.748130] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.748135] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748144] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.748155] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.748157] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.748159] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.748161] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.748189] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657748168), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.748196] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.748203] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.748207] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2258, ddl_arg=NULL) [2024-09-11 06:20:57.850356] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43598373) [2024-09-11 06:20:57.851145] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5222, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.851303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657850698000, start_id=1726035657850698000, end_id=1726035657850698001) [2024-09-11 06:20:57.851731] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.851746] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.851754] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.852432] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852458] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852472] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.852484] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.852489] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.852494] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852504] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852515] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.852518] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.852524] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.852527] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.852530] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852542] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.852553] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.852555] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.852557] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.852560] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.852580] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657852563), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.852587] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.852594] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.852597] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] [DDL] execute ddl like stmt(ret=-4122, cost=2243, ddl_arg=NULL) [2024-09-11 06:20:57.954823] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43493919) [2024-09-11 06:20:57.955582] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5223, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:57.955749] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035657954782000, start_id=1726035657954782000, end_id=1726035657954782001) [2024-09-11 06:20:57.956168] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:57.956177] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:57.956183] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.956796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956824] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956840] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:57.956873] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] added async task to update rslist(force_update=false) [2024-09-11 06:20:57.956878] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:57.956885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956897] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956907] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:57.956910] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.956913] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.956916] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:57.956918] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:57.956938] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:57.956940] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:57.956942] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:57.956945] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:57.956966] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035657956950), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:57.956973] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:57.956980] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:57.956986] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2164, ddl_arg=NULL) [2024-09-11 06:20:58.059431] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43389299) [2024-09-11 06:20:58.060263] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5224, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.060412] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035658059895000, start_id=1726035658059895000, end_id=1726035658059895001) [2024-09-11 06:20:58.060852] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.060868] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.060877] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.061578] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.061605] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.061627] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.061637] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:58.061644] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.061648] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.061668] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658061651), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.061676] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.061682] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.163974] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43284760) [2024-09-11 06:20:58.164751] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5225, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.164917] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035658164014000, start_id=1726035658164014000, end_id=1726035658164014001) [2024-09-11 06:20:58.165333] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.165346] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.165354] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.166010] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.166037] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.166052] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.166061] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.166068] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.166076] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.166093] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.166108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.166116] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:58.166120] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.166129] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.166157] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658166136), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.166166] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.166176] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.268699] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43180022) [2024-09-11 06:20:58.269517] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5226, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.269682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035658269244000, start_id=1726035658269244000, end_id=1726035658269244001) [2024-09-11 06:20:58.270114] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:58.270798] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.270833] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.270859] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.270873] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.270882] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.270886] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.270908] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658270889), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.270919] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.383693] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=43065033) [2024-09-11 06:20:58.384516] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5227, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.384671] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035658383891000, start_id=1726035658383891000, end_id=1726035658383891001) [2024-09-11 06:20:58.385064] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.385080] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.385092] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.385760] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.385798] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.385815] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.385826] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.385833] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.385838] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.385848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.385860] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.385863] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.385866] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.385872] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:58.385875] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.385888] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.385909] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658385892), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.385914] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.385920] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.488352] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42960377) [2024-09-11 06:20:58.489176] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5228, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.489339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035658489183000, start_id=1726035658489183000, end_id=1726035658489183001) [2024-09-11 06:20:58.489788] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.489806] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.489814] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.490510] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490555] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.490566] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.490572] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.490577] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490589] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490600] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.490604] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.490610] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.490613] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:58.490617] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.490641] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:58.490643] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.490647] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.490650] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:58.490675] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658490654), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.490683] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.490689] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.490695] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2345, ddl_arg=NULL) [2024-09-11 06:20:58.604173] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5229, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.604349] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035658603968000, start_id=1726035658603968000, end_id=1726035658603968001) [2024-09-11 06:20:58.604825] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.604838] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.604848] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.605610] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.605625] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.605655] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658605635), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.605661] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.605670] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.706881] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42741849) [2024-09-11 06:20:58.707675] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5230, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.707874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035658707251000, start_id=1726035658707251000, end_id=1726035658707251001) [2024-09-11 06:20:58.708288] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.708308] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.708317] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.708998] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709045] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.709059] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.709064] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.709073] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709103] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.709107] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.709113] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.709120] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:58.709125] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709140] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.709153] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:58.709157] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.709161] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.709166] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:58.709193] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658709173), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.709200] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.709209] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.709216] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2336, ddl_arg=NULL) [2024-09-11 06:20:58.811497] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42637238) [2024-09-11 06:20:58.812345] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5231, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.812568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035658812468000, start_id=1726035658812468000, end_id=1726035658812468001) [2024-09-11 06:20:58.812634] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=38, avg_log_batch_cnt=1, total_group_log_size=30361, avg_group_log_size=798) [2024-09-11 06:20:58.812645] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=30173) [2024-09-11 06:20:58.812986] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.812998] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.813004] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.813670] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813698] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813714] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.813724] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.813730] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.813735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813747] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813759] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.813762] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.813781] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.813786] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:58.813790] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813805] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.813816] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:58.813819] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.813822] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.813824] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:58.813846] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658813827), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.813854] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.813862] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.813867] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2372, ddl_arg=NULL) [2024-09-11 06:20:58.915980] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42532747) [2024-09-11 06:20:58.916762] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5232, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:58.916972] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1726035658916772000, start_id=1726035658916772000, end_id=1726035658916772001) [2024-09-11 06:20:58.917403] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:58.917414] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:58.917421] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.918071] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918100] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918116] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:58.918129] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:58.918136] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:58.918150] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918162] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918174] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:58.918177] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.918180] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.918184] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:58.918189] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918201] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:58.918213] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:58.918215] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:58.918218] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:58.918220] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:58.918244] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035658918226), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:58.918251] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:58.918260] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:58.918264] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2286, ddl_arg=NULL) [2024-09-11 06:20:59.020414] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42428309) [2024-09-11 06:20:59.021263] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5233, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.021461] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035659020952000, start_id=1726035659020952000, end_id=1726035659020952001) [2024-09-11 06:20:59.021921] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.021935] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.021944] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.022583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657068932, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022635] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.022650] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.022657] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.022663] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022686] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.022689] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.022693] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.022696] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.022700] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022711] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.022721] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.022724] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.022726] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.022729] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.022754] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659022732), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.022764] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.022772] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.022786] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] [DDL] execute ddl like stmt(ret=-4122, cost=2374, ddl_arg=NULL) [2024-09-11 06:20:59.125114] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42323617) [2024-09-11 06:20:59.125881] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5234, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.126058] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035659125404000, start_id=1726035659125404000, end_id=1726035659125404001) [2024-09-11 06:20:59.126482] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.126492] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.126500] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.127168] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127197] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127212] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.127224] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.127230] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.127236] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127249] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127259] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.127262] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.127265] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.127268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.127273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127284] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.127295] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.127297] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.127299] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.127301] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.127323] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659127304), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.127330] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.127338] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.127341] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2228, ddl_arg=NULL) [2024-09-11 06:20:59.229516] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42219219) [2024-09-11 06:20:59.230283] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5235, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.230432] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035659230341000, start_id=1726035659230341000, end_id=1726035659230341001) [2024-09-11 06:20:59.230902] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.230915] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.230921] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.231592] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231625] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231646] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.231663] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.231670] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.231675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231694] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657291806, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231706] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.231708] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.231711] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.231714] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.231717] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035657280466, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.231738] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.231740] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.231743] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.231745] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.231768] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659231748), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.231785] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.231791] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.231795] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2281, ddl_arg=NULL) [2024-09-11 06:20:59.333963] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42114781) [2024-09-11 06:20:59.334704] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5236, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.334898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035659334372000, start_id=1726035659334372000, end_id=1726035659334372001) [2024-09-11 06:20:59.335264] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:20:59.335275] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.335284] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.335292] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.335955] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.335983] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.335998] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.336008] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.336015] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.336022] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.336033] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.336045] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.336048] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.336053] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.336059] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.336063] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.336075] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.336086] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.336089] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.336091] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.336094] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.336115] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659336098), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.336121] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.336129] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.336133] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2172, ddl_arg=NULL) [2024-09-11 06:20:59.438330] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=42010392) [2024-09-11 06:20:59.439163] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:20:59.439810] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.439827] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.439836] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.440518] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.440530] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.440556] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659440535), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.440565] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.440572] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.553432] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41895290) [2024-09-11 06:20:59.554278] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=72] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5238, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.554434] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035659553437000, start_id=1726035659553437000, end_id=1726035659553437001) [2024-09-11 06:20:59.554854] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.554867] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.554875] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.555508] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555534] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555549] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.555560] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.555565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.555572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555584] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555595] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.555597] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.555602] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.555605] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.555608] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.555629] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.555631] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.555633] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.555636] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.555654] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659555639), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.555661] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.555666] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.555672] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2242, ddl_arg=NULL) [2024-09-11 06:20:59.658071] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41790666) [2024-09-11 06:20:59.658824] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5239, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.658988] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035659658607000, start_id=1726035659658607000, end_id=1726035659658607001) [2024-09-11 06:20:59.659406] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.659417] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.659425] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.660072] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660099] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660112] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.660125] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.660130] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.660137] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660147] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660158] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.660161] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.660164] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.660167] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.660172] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660184] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.660195] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.660199] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.660202] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.660205] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.660228] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659660209), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.660240] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.660247] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.660256] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2186, ddl_arg=NULL) [2024-09-11 06:20:59.762205] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41686534) [2024-09-11 06:20:59.763069] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5240, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.763252] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035659762952000, start_id=1726035659762952000, end_id=1726035659762952001) [2024-09-11 06:20:59.763666] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.763675] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.763681] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.764360] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.764371] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.764396] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659764378), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.764404] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.764412] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.866796] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41581938) [2024-09-11 06:20:59.867542] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5241, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.867687] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035659867061000, start_id=1726035659867061000, end_id=1726035659867061001) [2024-09-11 06:20:59.867755] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=36, avg_log_batch_cnt=1, total_group_log_size=28804, avg_group_log_size=800) [2024-09-11 06:20:59.867769] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28803) [2024-09-11 06:20:59.868109] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.868121] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.868128] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.868765] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868819] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=53] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868833] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.868846] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.868850] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.868856] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868867] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868876] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.868879] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.868884] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.868887] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.868890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868899] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.868910] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.868912] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.868914] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.868916] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.868937] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659868919), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.868944] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.868951] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.868955] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2161, ddl_arg=NULL) [2024-09-11 06:20:59.971323] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41477399) [2024-09-11 06:20:59.972145] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5242, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:20:59.972312] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035659971303000, start_id=1726035659971303000, end_id=1726035659971303001) [2024-09-11 06:20:59.972731] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:20:59.972744] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:20:59.972752] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.973426] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973476] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:20:59.973486] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:20:59.973492] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:20:59.973497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973509] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973519] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:20:59.973522] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.973530] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.973533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:20:59.973537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:20:59.973559] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:20:59.973562] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:20:59.973566] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:20:59.973571] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:20:59.973604] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035659973579), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:20:59.973615] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:20:59.973622] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:20:59.973629] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2307, ddl_arg=NULL) [2024-09-11 06:21:00.075779] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41372944) [2024-09-11 06:21:00.076551] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5243, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.076709] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035660076660000, start_id=1726035660076660000, end_id=1726035660076660001) [2024-09-11 06:21:00.077134] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.077148] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.077157] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.077793] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077822] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077838] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.077850] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.077857] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.077863] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077874] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077886] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.077889] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.077892] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.077896] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.077899] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077912] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.077923] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.077926] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.077930] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.077933] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.077954] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660077938), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.077963] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.077971] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.077975] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2204, ddl_arg=NULL) [2024-09-11 06:21:00.180847] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41267893) [2024-09-11 06:21:00.181650] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5244, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.181874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035660181834000, start_id=1726035660181834000, end_id=1726035660181834001) [2024-09-11 06:21:00.182291] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.182305] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.182313] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.182995] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183042] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.183055] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.183060] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.183067] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183090] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.183093] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.183099] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.183102] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.183108] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183119] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.183129] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.183131] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.183133] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.183136] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.183161] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660183142), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.183169] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.183174] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.183178] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2333, ddl_arg=NULL) [2024-09-11 06:21:00.285804] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41162943) [2024-09-11 06:21:00.286568] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5245, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.286760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035660286169000, start_id=1726035660286169000, end_id=1726035660286169001) [2024-09-11 06:21:00.287170] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.287185] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.287194] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.287910] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.287944] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.287965] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.287976] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.287982] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.287988] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.288005] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.288017] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.288019] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.288023] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.288027] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.288030] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.288041] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.288052] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.288055] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.288059] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.288063] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.288089] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660288069), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.288098] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.288104] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.288110] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2308, ddl_arg=NULL) [2024-09-11 06:21:00.390604] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=41058116) [2024-09-11 06:21:00.391398] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5246, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.391564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035660391335000, start_id=1726035660391335000, end_id=1726035660391335001) [2024-09-11 06:21:00.392050] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:00.392061] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.392069] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.392078] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.392763] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392806] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=42] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392820] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.392830] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.392835] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.392842] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392853] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392864] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.392866] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.392870] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.392875] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.392878] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392889] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.392904] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.392906] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.392909] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.392911] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.392935] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660392917), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.392943] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.392951] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.392957] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2355, ddl_arg=NULL) [2024-09-11 06:21:00.495318] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40953417) [2024-09-11 06:21:00.496178] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5247, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.496360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035660495642000, start_id=1726035660495642000, end_id=1726035660495642001) [2024-09-11 06:21:00.496819] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:00.497502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497530] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497546] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.497559] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.497565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.497572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497584] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497600] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.497603] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.497607] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.497611] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.497613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497625] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.497636] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.497639] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.497641] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.497643] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.497669] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660497649), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.497678] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.497686] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.497691] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2375, ddl_arg=NULL) [2024-09-11 06:21:00.600042] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40848688) [2024-09-11 06:21:00.600857] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5248, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:00.601043] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035660600745000, start_id=1726035660600745000, end_id=1726035660600745001) [2024-09-11 06:21:00.601473] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.601485] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.601494] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.602165] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602194] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602208] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.602223] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:00.602229] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:00.602234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602246] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602258] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:00.602261] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.602264] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.602267] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:00.602270] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.602291] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:00.602295] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:00.602298] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.602301] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:00.602324] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660602305), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.602334] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.602344] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:00.602351] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2310, ddl_arg=NULL) [2024-09-11 06:21:00.851981] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40596762) [2024-09-11 06:21:00.853000] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1726035660852022000, start_id=1726035660852022000, end_id=1726035660852022001) [2024-09-11 06:21:00.853518] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:00.853532] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:00.853540] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.854259] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.854293] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:00.854308] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:00.854332] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.854340] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:00.854362] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035660854343), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:00.854370] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:00.854382] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.004358] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40444382) [2024-09-11 06:21:01.005175] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5250, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.005359] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035661004587000, start_id=1726035661004587000, end_id=1726035661004587001) [2024-09-11 06:21:01.005798] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.005814] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.005823] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.006498] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006528] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659069281, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006543] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.006555] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.006561] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.006568] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006582] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006597] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.006599] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.006605] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:01.006609] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.006612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.006634] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.006637] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.006640] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:01.006643] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.006669] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661006649), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.006677] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.006685] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.006691] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2335, ddl_arg=NULL) [2024-09-11 06:21:01.109251] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40339484) [2024-09-11 06:21:01.110140] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5251, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.110318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035661109861000, start_id=1726035661109861000, end_id=1726035661109861001) [2024-09-11 06:21:01.110726] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.110740] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.110749] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.111427] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111458] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111479] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.111495] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.111502] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.111507] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111518] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111527] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.111530] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.111534] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.111537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.111560] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.111563] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.111565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.111594] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661111569), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.111601] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.111612] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.111619] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2369, ddl_arg=NULL) [2024-09-11 06:21:01.226896] INFO [RS] add_server (ob_root_service.cpp:7915) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40221842) [2024-09-11 06:21:01.227754] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5252, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.227963] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035661227085000, start_id=1726035661227085000, end_id=1726035661227085001) [2024-09-11 06:21:01.228425] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.228438] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.228447] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.229104] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229134] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.229161] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.229165] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.229170] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229183] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659293994, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229192] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.229195] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.229199] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.229205] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.229209] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229220] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035659280851, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.229231] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.229234] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.229237] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.229264] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661229242), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.229272] INFO [RS] add_server (ob_root_service.cpp:7928) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.229278] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.229282] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55201][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2389, ddl_arg=NULL) [2024-09-11 06:21:01.330682] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40118046) [2024-09-11 06:21:01.331474] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5253, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.331635] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035661331358000, start_id=1726035661331358000, end_id=1726035661331358001) [2024-09-11 06:21:01.332087] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.332102] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.332111] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.332832] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332867] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332883] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.332895] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.332901] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.332908] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332923] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332937] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.332939] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.332942] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.332948] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.332951] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332961] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.332979] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.332983] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.332989] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.333014] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661332994), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.333022] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.333028] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.333034] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2354, ddl_arg=NULL) [2024-09-11 06:21:01.435457] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=40013285) [2024-09-11 06:21:01.436281] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5254, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.436455] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035661435445000, start_id=1726035661435445000, end_id=1726035661435445001) [2024-09-11 06:21:01.436848] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:01.436860] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.436869] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.436876] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.437533] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.437563] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.437579] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.437591] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.437595] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.437603] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.437614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.437644] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661437626), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.437651] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.437656] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.540202] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39908523) [2024-09-11 06:21:01.540945] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5255, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.541111] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035661540672000, start_id=1726035661540672000, end_id=1726035661540672001) [2024-09-11 06:21:01.541529] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.541542] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.541551] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.542302] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542335] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542351] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.542362] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.542368] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.542373] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542396] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.542398] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.542403] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.542410] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.542414] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542426] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.542437] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.542440] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.542443] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.542464] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661542446), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.542472] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.542479] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.542485] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2284, ddl_arg=NULL) [2024-09-11 06:21:01.644983] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39803751) [2024-09-11 06:21:01.645755] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5256, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.645963] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035661645949000, start_id=1726035661645949000, end_id=1726035661645949001) [2024-09-11 06:21:01.646377] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.646390] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.646398] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.647072] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.647099] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.647112] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.647123] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.647128] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.647139] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.647149] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.647179] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661647163), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.647185] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.647190] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.749785] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39698947) [2024-09-11 06:21:01.750514] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5257, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.750683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035661750000000, start_id=1726035661750000000, end_id=1726035661750000001) [2024-09-11 06:21:01.751100] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.751113] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.751121] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.751786] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751826] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751843] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.751855] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.751861] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.751866] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751885] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.751888] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.751891] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.751894] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.751897] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.751917] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.751919] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.751922] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.751941] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661751925), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.751948] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.751955] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.751959] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2175, ddl_arg=NULL) [2024-09-11 06:21:01.854464] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39594268) [2024-09-11 06:21:01.855250] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5258, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.855422] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035661855344000, start_id=1726035661855344000, end_id=1726035661855344001) [2024-09-11 06:21:01.855856] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.855871] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.855881] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.856538] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.856566] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.856581] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.856591] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.856598] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.856603] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.856614] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:01.856655] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661856636), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.856664] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.856669] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.856676] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2213, ddl_arg=NULL) [2024-09-11 06:21:01.959069] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39489654) [2024-09-11 06:21:01.959900] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5259, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:01.960069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035661959421000, start_id=1726035661959421000, end_id=1726035661959421001) [2024-09-11 06:21:01.960493] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:01.960507] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:01.960515] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.961187] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961242] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:01.961254] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:01.961262] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:01.961269] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961281] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961292] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:01.961295] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.961299] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:01.961304] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961315] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:01.961326] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:01.961329] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:01.961331] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:01.961356] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035661961334), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:01.961370] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:01.961379] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:01.961383] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2317, ddl_arg=NULL) [2024-09-11 06:21:02.063674] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39385061) [2024-09-11 06:21:02.064446] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5260, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.064606] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035662063780000, start_id=1726035662063780000, end_id=1726035662063780001) [2024-09-11 06:21:02.065041] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.065067] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.065076] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.065721] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065764] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.065791] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.065797] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.065804] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065818] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065830] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.065833] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.065837] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:02.065841] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.065846] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065857] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.065869] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.065872] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.065874] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.065898] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662065879), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.065906] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.065912] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.065915] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2243, ddl_arg=NULL) [2024-09-11 06:21:02.168393] INFO [RS] add_server (ob_root_service.cpp:7915) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39280336) [2024-09-11 06:21:02.169150] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5261, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.169332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035662168908000, start_id=1726035662168908000, end_id=1726035662168908001) [2024-09-11 06:21:02.169758] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.169770] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.169802] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.170425] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170452] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170466] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.170478] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.170484] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.170491] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170512] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.170515] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.170518] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:02.170523] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.170526] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.170548] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.170550] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.170552] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.170572] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662170556), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.170577] INFO [RS] add_server (ob_root_service.cpp:7928) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.170584] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.170590] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2199, ddl_arg=NULL) [2024-09-11 06:21:02.272998] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39175734) [2024-09-11 06:21:02.273764] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5262, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.273972] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=48] get number(ret=0, service_type_=0, range=1, base_id=1726035662273059000, start_id=1726035662273059000, end_id=1726035662273059001) [2024-09-11 06:21:02.274370] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.274381] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.274387] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.275033] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275062] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275079] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.275090] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.275093] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.275098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275110] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275121] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.275123] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.275127] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.275130] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275140] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.275151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.275154] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.275156] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.275180] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662275160), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.275187] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.275195] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.275200] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2204, ddl_arg=NULL) [2024-09-11 06:21:02.377523] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=39071212) [2024-09-11 06:21:02.378590] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5263, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.378791] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035662378412000, start_id=1726035662378412000, end_id=1726035662378412001) [2024-09-11 06:21:02.379200] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.379213] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.379222] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.379918] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.379946] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.379962] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.379971] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.379977] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.379983] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.379994] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.380005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.380009] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.380015] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:02.380019] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.380022] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.380034] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.380051] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.380054] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.380057] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.380084] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662380060), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.380093] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.380100] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.380107] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2586, ddl_arg=NULL) [2024-09-11 06:21:02.482709] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:02.483485] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5264, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.483652] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035662483632000, start_id=1726035662483632000, end_id=1726035662483632001) [2024-09-11 06:21:02.484104] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:02.484119] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.484128] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.484135] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.484795] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.484826] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.484840] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.484852] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.484859] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.484884] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662484863), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.484892] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.484903] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.587455] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38861275) [2024-09-11 06:21:02.588195] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5265, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.588356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035662587739000, start_id=1726035662587739000, end_id=1726035662587739001) [2024-09-11 06:21:02.588805] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.589497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589530] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589545] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.589555] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.589561] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.589568] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589579] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589591] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.589593] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.589597] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.589602] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.589605] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589617] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.589628] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.589630] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.589635] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.589638] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.589662] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662589641), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.589671] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.589681] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2228, ddl_arg=NULL) [2024-09-11 06:21:02.692274] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38756456) [2024-09-11 06:21:02.693086] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5266, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.693259] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035662692831000, start_id=1726035662692831000, end_id=1726035662692831001) [2024-09-11 06:21:02.693700] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.693712] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.693721] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.694458] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694496] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694517] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.694529] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.694537] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.694541] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694552] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694563] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.694566] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.694569] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.694571] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.694574] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694584] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.694595] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.694597] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.694600] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.694602] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.694622] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662694605), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.694630] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.694637] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.694641] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2369, ddl_arg=NULL) [2024-09-11 06:21:02.797038] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38651712) [2024-09-11 06:21:02.797821] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5267, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.798015] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035662797014000, start_id=1726035662797014000, end_id=1726035662797014001) [2024-09-11 06:21:02.798494] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.798507] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.798517] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.799204] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799257] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.799268] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.799274] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.799280] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799291] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799302] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.799306] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.799310] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.799314] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.799319] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799330] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.799342] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.799345] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.799348] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.799351] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.799376] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662799356), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.799384] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.799390] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.799396] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2360, ddl_arg=NULL) [2024-09-11 06:21:02.901824] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38546921) [2024-09-11 06:21:02.902658] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5268, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:02.902908] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:157) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] submit log success(service_type_=0, last_id=1726035679537832000, limited_id=1726035679537832000) [2024-09-11 06:21:02.902918] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] submit log with lock success(service_type_=0, last_id=1726035679537832000, limited_id=1726035679537832000) [2024-09-11 06:21:02.902922] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1726035662902172000, start_id=1726035662902172000, end_id=1726035662902172001) [2024-09-11 06:21:02.903441] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:02.903458] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:02.903466] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.904165] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904195] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904209] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:02.904219] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:02.904225] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:02.904230] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904242] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904254] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:02.904257] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.904261] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.904264] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:02.904268] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904279] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:02.904288] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:02.904291] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:02.904294] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:02.904296] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:02.904321] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035662904300), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:02.904330] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:02.904336] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:02.904340] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2518, ddl_arg=NULL) [2024-09-11 06:21:03.006464] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38442277) [2024-09-11 06:21:03.007280] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5269, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.007465] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663007349000, start_id=1726035663007349000, end_id=1726035663007349001) [2024-09-11 06:21:03.007918] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.007932] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.007940] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.008593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008626] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661069619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008647] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.008662] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.008668] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.008675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008698] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.008701] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.008704] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.008710] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.008713] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008722] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.008733] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.008736] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.008738] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.008740] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.008764] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663008743), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.008800] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.008808] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.008815] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2354, ddl_arg=NULL) [2024-09-11 06:21:03.111116] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38337613) [2024-09-11 06:21:03.111888] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5270, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.112073] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035663111680000, start_id=1726035663111680000, end_id=1726035663111680001) [2024-09-11 06:21:03.112522] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.112536] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.112544] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.113256] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113290] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113307] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.113318] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.113323] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.113331] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113353] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.113355] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.113358] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.113362] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.113367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.113388] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.113390] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.113392] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.113394] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.113415] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663113398), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.113422] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.113430] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.113434] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2320, ddl_arg=NULL) [2024-09-11 06:21:03.216333] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38232397) [2024-09-11 06:21:03.217145] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5271, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.217351] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663216944000, start_id=1726035663216944000, end_id=1726035663216944001) [2024-09-11 06:21:03.217813] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.217825] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.217831] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.218497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218525] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218541] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.218553] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.218560] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.218565] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661294641, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218587] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.218590] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.218594] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.218601] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.218604] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218616] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035661281267, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.218627] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.218630] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.218633] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.218637] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.218659] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663218641), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.218667] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.218675] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.218679] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2349, ddl_arg=NULL) [2024-09-11 06:21:03.320473] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38128269) [2024-09-11 06:21:03.321586] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:03.322240] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.322256] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.322265] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.322964] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.322976] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.322982] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.323005] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663322988), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.323015] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.323024] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.425875] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=38022857) [2024-09-11 06:21:03.426655] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5273, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.426850] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663426221000, start_id=1726035663426221000, end_id=1726035663426221001) [2024-09-11 06:21:03.427331] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.427344] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.427351] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.428108] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428148] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428171] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.428184] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.428192] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.428200] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428229] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428247] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.428254] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.428259] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.428267] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.428277] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428296] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.428315] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.428319] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.428325] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.428329] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.428358] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663428337), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.428367] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.428384] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.428393] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2520, ddl_arg=NULL) [2024-09-11 06:21:03.531985] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37916747) [2024-09-11 06:21:03.532808] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5274, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.532986] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663532436000, start_id=1726035663532436000, end_id=1726035663532436001) [2024-09-11 06:21:03.533485] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:03.533495] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.533504] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.533512] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.534177] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534207] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534223] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.534235] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.534241] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.534246] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534258] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534270] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.534273] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.534277] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.534280] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.534286] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534297] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.534307] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.534313] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.534315] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.534318] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.534341] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663534322), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.534349] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.534355] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.534361] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2379, ddl_arg=NULL) [2024-09-11 06:21:03.636934] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:03.638349] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.639076] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.639088] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.639116] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663639093), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.639126] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.741680] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37707051) [2024-09-11 06:21:03.742458] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5276, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.742626] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663741993000, start_id=1726035663741993000, end_id=1726035663741993001) [2024-09-11 06:21:03.743046] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.743059] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.743066] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.743705] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743734] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743751] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.743761] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.743767] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.743772] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743792] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743804] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.743807] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.743810] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.743814] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.743820] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743832] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.743847] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:03.743850] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.743852] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.743854] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:03.743879] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663743860), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.743887] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.743893] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.743897] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2218, ddl_arg=NULL) [2024-09-11 06:21:03.846440] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:03.847233] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5277, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.847403] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035663847176000, start_id=1726035663847176000, end_id=1726035663847176001) [2024-09-11 06:21:03.847818] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.847830] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.847839] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.848492] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.848534] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.848542] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.848630] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663848546), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.848637] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.848644] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:03.951612] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37497127) [2024-09-11 06:21:03.952395] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5278, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:03.952599] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035663952397000, start_id=1726035663952397000, end_id=1726035663952397001) [2024-09-11 06:21:03.952669] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=33, avg_log_batch_cnt=1, total_group_log_size=26499, avg_group_log_size=803) [2024-09-11 06:21:03.952681] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=26310) [2024-09-11 06:21:03.953031] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:03.953044] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:03.953053] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.953685] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953713] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953726] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:03.953738] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:03.953744] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:03.953749] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953761] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953787] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:03.953790] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:03.953793] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.953797] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:03.953802] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953812] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:03.953822] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:03.953825] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:03.953849] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035663953830), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:03.953857] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:03.953866] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.056742] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37391987) [2024-09-11 06:21:04.058074] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.058090] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.058098] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.058745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.058801] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.058809] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.058834] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664058814), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.058843] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.058850] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.161956] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37286779) [2024-09-11 06:21:04.162718] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5280, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.162949] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035664162676000, start_id=1726035664162676000, end_id=1726035664162676001) [2024-09-11 06:21:04.163379] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.163392] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.163401] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.164055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164085] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164106] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:04.164116] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.164123] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:04.164128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164140] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:04.164154] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:04.164157] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.164160] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:04.164164] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164176] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.164192] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:04.164196] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.164217] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664164199), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.164225] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.164231] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.267350] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37181383) [2024-09-11 06:21:04.268120] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5281, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.268274] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035664268017000, start_id=1726035664268017000, end_id=1726035664268017001) [2024-09-11 06:21:04.268681] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.268693] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.268701] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.269385] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.269394] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.269416] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664269398), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.269424] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.269432] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.372500] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=37076234) [2024-09-11 06:21:04.373303] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5282, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.373492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035664373395000, start_id=1726035664373395000, end_id=1726035664373395001) [2024-09-11 06:21:04.373940] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.373953] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.373960] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.374600] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374629] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374645] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:04.374655] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.374662] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:04.374668] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374680] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374696] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:04.374700] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:04.374705] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.374709] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:04.374715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.374739] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:04.374741] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:04.374745] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.374748] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:04.374770] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664374752), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.374790] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.374798] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.374807] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2307, ddl_arg=NULL) [2024-09-11 06:21:04.478028] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36970753) [2024-09-11 06:21:04.479129] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5283, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.479364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035664478661000, start_id=1726035664478661000, end_id=1726035664478661001) [2024-09-11 06:21:04.479860] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.479875] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.479884] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.480553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:04.480594] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.480603] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.480625] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664480607), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.480636] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.480646] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.583988] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36864751) [2024-09-11 06:21:04.585090] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5284, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.585291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035664585077000, start_id=1726035664585077000, end_id=1726035664585077001) [2024-09-11 06:21:04.585868] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:04.585883] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.585894] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.585901] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.586810] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.586824] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.586831] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.586859] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664586835), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.586871] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.586884] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:04.689442] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36759287) [2024-09-11 06:21:04.690255] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5285, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.690418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035664690268000, start_id=1726035664690268000, end_id=1726035664690268001) [2024-09-11 06:21:04.690899] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.690913] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:04.691594] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.691623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.691639] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:04.691650] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.691657] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:04.691662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.691675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.691689] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.691695] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.691725] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664691703), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.691738] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.795087] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36653652) [2024-09-11 06:21:04.795935] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5286, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.796145] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035664795255000, start_id=1726035664795255000, end_id=1726035664795255001) [2024-09-11 06:21:04.796577] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:04.797290] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.797317] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.797339] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.797347] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.797355] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.797375] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664797358), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.797383] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.900539] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36548196) [2024-09-11 06:21:04.901382] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5287, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:04.901574] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035664900704000, start_id=1726035664900704000, end_id=1726035664900704001) [2024-09-11 06:21:04.902018] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:04.902029] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:04.902037] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.902687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.902716] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:04.902730] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:04.902741] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:21:04.902749] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:04.902753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:04.902768] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.902772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:04.902808] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035664902790), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:04.902816] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:04.902821] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.005893] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36442843) [2024-09-11 06:21:05.006679] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5288, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.006887] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035665006052000, start_id=1726035665006052000, end_id=1726035665006052001) [2024-09-11 06:21:05.006965] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=36, avg_log_batch_cnt=1, total_group_log_size=28800, avg_group_log_size=800) [2024-09-11 06:21:05.006977] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28800) [2024-09-11 06:21:05.007353] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.007366] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.007373] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.008031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008060] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663069934, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008077] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.008088] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.008095] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.008100] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008123] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.008126] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.008130] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.008134] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.008139] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008152] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.008164] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.008167] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.008171] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.008175] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.008215] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665008179), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.008227] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.008238] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.008245] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2353, ddl_arg=NULL) [2024-09-11 06:21:05.111035] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36337690) [2024-09-11 06:21:05.111869] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5289, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.112048] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035665111349000, start_id=1726035665111349000, end_id=1726035665111349001) [2024-09-11 06:21:05.112509] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.112523] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.112532] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.113185] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113214] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113229] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.113239] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.113244] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.113251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113262] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113274] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.113277] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.113281] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.113285] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.113288] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113299] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.113309] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.113316] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.113319] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.113322] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.113348] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665113326), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.113358] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.113367] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.113372] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2339, ddl_arg=NULL) [2024-09-11 06:21:05.216248] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36232475) [2024-09-11 06:21:05.217044] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5290, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.217223] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1726035665216787000, start_id=1726035665216787000, end_id=1726035665216787001) [2024-09-11 06:21:05.217671] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.217684] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.217692] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.218342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218378] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218394] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.218421] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.218428] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.218433] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218444] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663295367, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218456] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.218459] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.218463] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.218466] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.218469] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218481] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035663281669, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.218492] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:05.218496] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.218518] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665218500), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.218527] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.218533] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.321368] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36127356) [2024-09-11 06:21:05.322164] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5291, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.322328] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035665322114000, start_id=1726035665322114000, end_id=1726035665322114001) [2024-09-11 06:21:05.322746] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.322759] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.322769] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.323470] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.323500] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:05.323531] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.323539] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.323565] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665323544), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.323576] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.323586] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.426338] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=36022390) [2024-09-11 06:21:05.427103] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5292, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.427285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035665426268000, start_id=1726035665426268000, end_id=1726035665426268001) [2024-09-11 06:21:05.427707] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.427719] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.427728] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.428386] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428415] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428431] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.428444] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.428450] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.428455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428467] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428480] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.428483] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.428488] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.428492] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.428503] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428514] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.428525] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.428528] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.428531] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.428533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.428554] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665428536), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.428562] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.428574] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.428578] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2241, ddl_arg=NULL) [2024-09-11 06:21:05.531694] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35917047) [2024-09-11 06:21:05.532494] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5293, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.532674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035665532544000, start_id=1726035665532544000, end_id=1726035665532544001) [2024-09-11 06:21:05.533099] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.533112] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.533121] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.533817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533847] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533863] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.533876] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.533882] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.533889] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533901] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533912] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.533920] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.533924] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.533929] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.533932] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533944] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.533955] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.533957] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.533959] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.533964] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.533987] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665533968), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.533994] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.534001] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.534008] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2316, ddl_arg=NULL) [2024-09-11 06:21:05.636656] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35812082) [2024-09-11 06:21:05.637422] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5294, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.637583] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035665637551000, start_id=1726035665637551000, end_id=1726035665637551001) [2024-09-11 06:21:05.638037] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:05.638048] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.638056] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.638064] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.638698] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.638726] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.638742] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.638754] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.638761] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.638766] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.638807] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.638818] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.638824] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.638841] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.638846] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.638854] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:05.638870] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.638892] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665638874), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.638900] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.638915] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.741532] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35707202) [2024-09-11 06:21:05.742310] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5295, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.742470] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035665741481000, start_id=1726035665741481000, end_id=1726035665741481001) [2024-09-11 06:21:05.742883] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.742895] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.742902] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.743548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743574] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743599] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.743613] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.743619] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.743627] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743648] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743662] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.743666] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.743671] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.743675] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.743683] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743698] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.743711] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.743716] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.743721] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.743725] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.743749] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665743730), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.743756] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.743767] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.743772] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2241, ddl_arg=NULL) [2024-09-11 06:21:05.846195] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35602544) [2024-09-11 06:21:05.846997] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5296, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.847175] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035665846705000, start_id=1726035665846705000, end_id=1726035665846705001) [2024-09-11 06:21:05.847575] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.847588] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:05.848275] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848305] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848322] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.848342] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.848350] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.848355] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848377] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.848383] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.848388] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.848391] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.848397] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848418] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.848433] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.848438] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.848441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.848448] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.848476] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665848451), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.848497] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.848508] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2315, ddl_arg=NULL) [2024-09-11 06:21:05.951005] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35497726) [2024-09-11 06:21:05.951804] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5297, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:05.951977] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035665950949000, start_id=1726035665950949000, end_id=1726035665950949001) [2024-09-11 06:21:05.952366] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:05.952379] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:05.952386] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.953054] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953084] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953098] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:05.953110] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:05.953116] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:05.953122] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953133] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953144] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:05.953147] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.953151] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.953154] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:05.953157] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953170] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:05.953185] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:05.953191] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:05.953194] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:05.953197] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:05.953221] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035665953200), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:05.953230] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:05.953236] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:05.953242] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2239, ddl_arg=NULL) [2024-09-11 06:21:06.056267] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35392468) [2024-09-11 06:21:06.057093] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5298, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.057278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035666056328000, start_id=1726035666056328000, end_id=1726035666056328001) [2024-09-11 06:21:06.057339] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28258, avg_group_log_size=807) [2024-09-11 06:21:06.057350] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28258) [2024-09-11 06:21:06.057690] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.057702] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.057717] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.058436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058467] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058483] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.058494] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.058500] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.058505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058516] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058527] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.058531] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.058535] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.058539] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.058545] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058557] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.058568] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.058571] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.058574] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.058577] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.058598] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666058580), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.058607] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.058615] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.058620] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2355, ddl_arg=NULL) [2024-09-11 06:21:06.161281] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35287454) [2024-09-11 06:21:06.162034] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5299, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.162205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035666161745000, start_id=1726035666161745000, end_id=1726035666161745001) [2024-09-11 06:21:06.162661] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.162673] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.162680] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.163333] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163364] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163380] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.163393] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.163399] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.163405] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163416] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163427] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.163429] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.163435] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.163438] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.163441] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163453] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.163463] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.163466] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.163468] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.163471] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.163492] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666163474), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.163499] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.163509] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.163516] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2237, ddl_arg=NULL) [2024-09-11 06:21:06.265999] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35182735) [2024-09-11 06:21:06.266812] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5300, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.266990] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035666266003000, start_id=1726035666266003000, end_id=1726035666266003001) [2024-09-11 06:21:06.267408] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.267420] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.267429] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.268088] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268116] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268132] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.268144] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.268150] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.268157] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268169] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268180] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.268183] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.268186] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.268190] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.268196] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268207] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.268217] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.268219] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.268221] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.268224] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.268249] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666268230), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.268258] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.268268] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.268275] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:21:06.370607] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=35078126) [2024-09-11 06:21:06.371356] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5301, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.371558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035666371205000, start_id=1726035666371205000, end_id=1726035666371205001) [2024-09-11 06:21:06.372017] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.372030] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.372039] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.372684] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372719] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372735] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.372747] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.372754] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.372760] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372771] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372790] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.372794] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.372800] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.372805] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.372811] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.372833] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.372836] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.372838] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.372840] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.372864] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666372845), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.372870] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.372878] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.372883] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:21:06.475566] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34973195) [2024-09-11 06:21:06.476383] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:06.477125] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.477140] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.477148] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.477987] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.478000] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.478026] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666478005), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.478037] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.478044] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.580900] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34867857) [2024-09-11 06:21:06.581718] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5303, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.581935] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035666581597000, start_id=1726035666581597000, end_id=1726035666581597001) [2024-09-11 06:21:06.582417] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.582431] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.582441] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.583134] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583185] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.583197] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.583206] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.583215] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583232] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583248] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.583252] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.583256] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.583259] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583271] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.583281] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.583284] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.583286] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.583307] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666583289), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.583315] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.583321] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.583327] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2429, ddl_arg=NULL) [2024-09-11 06:21:06.685815] INFO [RS] add_server (ob_root_service.cpp:7915) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34762946) [2024-09-11 06:21:06.686644] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5304, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.686944] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035666686644000, start_id=1726035666686644000, end_id=1726035666686644001) [2024-09-11 06:21:06.687390] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:06.687401] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.687409] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.687417] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.688056] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688105] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.688116] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.688123] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.688128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688140] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.688155] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.688159] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:06.688166] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.688169] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.688193] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.688196] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.688199] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.688221] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666688204), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.688229] INFO [RS] add_server (ob_root_service.cpp:7928) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.688235] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.688241] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55200][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2428, ddl_arg=NULL) [2024-09-11 06:21:06.790587] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34658147) [2024-09-11 06:21:06.791344] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5305, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.791510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035666790558000, start_id=1726035666790558000, end_id=1726035666790558001) [2024-09-11 06:21:06.791953] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.791965] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:06.791974] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.792645] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792688] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.792698] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.792704] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.792711] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792721] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792730] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.792733] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.792739] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.792742] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:06.792745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792756] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.792767] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:06.792770] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.792807] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.792811] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:06.792832] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666792815), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.792840] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:06.792846] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:06.792850] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2264, ddl_arg=NULL) [2024-09-11 06:21:06.895485] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34553247) [2024-09-11 06:21:06.896471] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5306, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:06.896678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035666895748000, start_id=1726035666895748000, end_id=1726035666895748001) [2024-09-11 06:21:06.897183] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:06.897199] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:06.898069] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.898104] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.898126] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:06.898141] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:06.898149] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:06.898157] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.898176] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:06.898191] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:06.898196] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:06.898204] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.898215] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:06.898241] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035666898220), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:06.898252] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.000597] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34448138) [2024-09-11 06:21:07.001360] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5307, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.001547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035667000962000, start_id=1726035667000962000, end_id=1726035667000962001) [2024-09-11 06:21:07.002666] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002697] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665070283, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002711] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.002730] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.002736] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.002741] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002765] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.002768] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.002772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.002783] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.002787] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002800] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.002810] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.002812] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.002816] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.002821] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.002848] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667002827), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.002855] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.002864] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2269, ddl_arg=NULL) [2024-09-11 06:21:07.105464] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34343265) [2024-09-11 06:21:07.106312] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5308, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.106520] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035667106203000, start_id=1726035667106203000, end_id=1726035667106203001) [2024-09-11 06:21:07.106595] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28259, avg_group_log_size=807) [2024-09-11 06:21:07.106611] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28259) [2024-09-11 06:21:07.107000] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.107011] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.107021] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.107799] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107830] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107847] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.107861] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.107866] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.107871] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107884] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107895] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.107898] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.107902] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.107909] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.107915] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107932] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.107944] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.107947] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.107949] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.107954] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.107981] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667107957), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.107990] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.107998] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.108004] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2542, ddl_arg=NULL) [2024-09-11 06:21:07.210604] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34238135) [2024-09-11 06:21:07.211585] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5309, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.211760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035667211511000, start_id=1726035667211511000, end_id=1726035667211511001) [2024-09-11 06:21:07.212196] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.212211] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.212220] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.212906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.212937] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.212955] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.212966] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.212970] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.212978] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.212990] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665296205, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.213002] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.213005] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.213009] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.213015] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.213019] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.213031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035665282049, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.213048] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.213052] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.213054] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.213057] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.213090] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667213065), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.213100] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.213111] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.213120] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:07.315420] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34133305) [2024-09-11 06:21:07.316296] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5310, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.316486] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035667315786000, start_id=1726035667315786000, end_id=1726035667315786001) [2024-09-11 06:21:07.316922] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.316939] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.316949] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.317601] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317632] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317652] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.317668] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.317673] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.317679] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317696] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317713] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.317718] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.317724] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.317730] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.317736] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.317769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.317819] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=50] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.317823] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.317826] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.317855] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667317833), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.317863] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.317872] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.317880] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2461, ddl_arg=NULL) [2024-09-11 06:21:07.420101] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=34028622) [2024-09-11 06:21:07.420888] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5311, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.421061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035667420017000, start_id=1726035667420017000, end_id=1726035667420017001) [2024-09-11 06:21:07.421463] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.421478] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.421490] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.422167] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.422196] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.422212] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.422224] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.422230] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.422234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.422248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.422265] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.422269] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.422277] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.422282] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.422287] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:07.422299] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.422322] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667422303), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.422334] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.422343] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.525367] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33923366) [2024-09-11 06:21:07.526216] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5312, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.526385] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035667525477000, start_id=1726035667525477000, end_id=1726035667525477001) [2024-09-11 06:21:07.526832] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.526849] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.526855] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.527533] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527564] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527579] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.527590] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.527596] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.527601] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527624] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.527627] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.527631] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.527635] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.527641] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.527665] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.527669] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.527675] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.527679] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.527705] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667527684), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.527714] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.527720] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.527726] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2362, ddl_arg=NULL) [2024-09-11 06:21:07.629988] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33818737) [2024-09-11 06:21:07.630794] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5313, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.630982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035667630779000, start_id=1726035667630779000, end_id=1726035667630779001) [2024-09-11 06:21:07.631409] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.631421] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.631429] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.632106] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632136] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.632163] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.632169] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.632175] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632187] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632204] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.632208] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.632213] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.632217] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.632220] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632231] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.632242] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.632245] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.632248] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.632250] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.632274] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667632254), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.632284] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.632292] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.632298] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2312, ddl_arg=NULL) [2024-09-11 06:21:07.734596] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33714130) [2024-09-11 06:21:07.735341] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5314, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.735540] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035667734959000, start_id=1726035667734959000, end_id=1726035667734959001) [2024-09-11 06:21:07.735981] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:07.735992] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.735999] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.736007] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.736638] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736666] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736683] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.736695] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.736702] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.736706] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736717] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736729] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.736732] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.736738] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.736742] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:07.736745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.736766] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:07.736769] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.736771] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:07.736779] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:07.736800] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667736783), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.736808] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.736814] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.736820] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:07.838969] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:07.839781] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5315, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.839954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035667839072000, start_id=1726035667839072000, end_id=1726035667839072001) [2024-09-11 06:21:07.840406] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.840417] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.840426] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.841115] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667841094), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.841126] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.841135] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:07.943608] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33505118) [2024-09-11 06:21:07.944415] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5316, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:07.944591] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035667944293000, start_id=1726035667944293000, end_id=1726035667944293001) [2024-09-11 06:21:07.944995] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:07.945008] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:07.945016] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.945662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.945690] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.945710] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:07.945724] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:07.945731] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:07.945740] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.945752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:07.945763] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:07.945766] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:07.945772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:07.945812] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035667945790), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:07.945819] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:07.945826] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.047966] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33400772) [2024-09-11 06:21:08.048843] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5317, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.049055] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035668048332000, start_id=1726035668048332000, end_id=1726035668048332001) [2024-09-11 06:21:08.049484] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.050217] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050246] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050266] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.050277] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.050284] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.050288] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050302] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050314] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:08.050317] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.050322] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:08.050328] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050340] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.050350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:08.050354] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.050357] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:08.050378] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668050362), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.050384] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.050394] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2431, ddl_arg=NULL) [2024-09-11 06:21:08.163012] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33285721) [2024-09-11 06:21:08.163837] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5318, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.164020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035668162998000, start_id=1726035668162998000, end_id=1726035668162998001) [2024-09-11 06:21:08.164491] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.164505] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.164513] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.165206] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165237] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165253] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.165263] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.165268] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.165275] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165286] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165298] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:08.165308] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.165314] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:08.165318] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165330] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.165341] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:08.165344] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.165347] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:08.165372] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668165352), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.165381] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.165389] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.165393] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2383, ddl_arg=NULL) [2024-09-11 06:21:08.268009] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33180724) [2024-09-11 06:21:08.268765] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5319, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.268977] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=44] get number(ret=0, service_type_=0, range=1, base_id=1726035668268314000, start_id=1726035668268314000, end_id=1726035668268314001) [2024-09-11 06:21:08.269407] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.269421] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.269430] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.270111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.270146] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.270166] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.270177] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.270183] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.270188] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.270198] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.270232] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668270214), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.270240] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.270247] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.270253] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2244, ddl_arg=NULL) [2024-09-11 06:21:08.383331] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=33065398) [2024-09-11 06:21:08.384119] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5320, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.384285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:08.384698] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.384710] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.384719] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.385352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.385388] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.385411] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.385440] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668385423), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.385449] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.385455] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.488087] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:08.489473] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.489484] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.489492] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.490148] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.490206] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668490191), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.490212] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.490218] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.603210] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32845517) [2024-09-11 06:21:08.603980] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5322, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.604138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035668603996000, start_id=1726035668603996000, end_id=1726035668603996001) [2024-09-11 06:21:08.604548] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.604562] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.604571] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.605256] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605300] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.605312] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.605317] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.605324] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605335] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605346] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:08.605350] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.605356] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:08.605360] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:08.605363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605375] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.605385] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:08.605390] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.605393] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:08.605417] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668605397), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.605426] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.605433] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.707836] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32740919) [2024-09-11 06:21:08.708615] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5323, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.708789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035668708219000, start_id=1726035668708219000, end_id=1726035668708219001) [2024-09-11 06:21:08.709183] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.709196] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.709205] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.709884] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.709922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.709947] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.709963] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.709972] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.710013] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668709987), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.710025] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.710036] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.812286] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32636449) [2024-09-11 06:21:08.813120] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5324, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.813304] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035668813270000, start_id=1726035668813270000, end_id=1726035668813270001) [2024-09-11 06:21:08.813714] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:08.813724] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.813735] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.813742] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.814403] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814431] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814447] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.814459] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:08.814465] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:08.814472] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814483] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814494] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:08.814497] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.814500] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:08.814504] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:08.814507] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814516] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.814526] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:08.814529] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:08.814531] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:08.814533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:08.814557] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668814537), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.814565] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.814572] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:08.814578] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2295, ddl_arg=NULL) [2024-09-11 06:21:08.917165] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32531567) [2024-09-11 06:21:08.918020] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5325, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:08.918212] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035668917443000, start_id=1726035668917443000, end_id=1726035668917443001) [2024-09-11 06:21:08.918604] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:08.918617] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:08.918626] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.919353] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.919384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:08.919399] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:08.919434] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035668919415), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:08.919443] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:08.919450] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.021731] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32427011) [2024-09-11 06:21:09.022618] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5326, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.022809] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035669022714000, start_id=1726035669022714000, end_id=1726035669022714001) [2024-09-11 06:21:09.023237] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.023249] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.023258] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.023953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.023988] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667070630, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.024005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.024015] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.024021] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.024028] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.024040] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.024052] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.024054] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.024061] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.024065] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.024076] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.024087] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.024090] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.024092] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.024117] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669024096), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.024125] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.024134] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.024140] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2412, ddl_arg=NULL) [2024-09-11 06:21:09.126430] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32322295) [2024-09-11 06:21:09.127235] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5327, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.127401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035669126898000, start_id=1726035669126898000, end_id=1726035669126898001) [2024-09-11 06:21:09.127824] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.127839] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.127848] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.128502] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128533] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128548] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.128560] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.128575] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.128580] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128603] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.128606] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.128609] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:09.128613] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.128616] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128628] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.128637] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.128639] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.128643] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.128665] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669128646), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.128673] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.128680] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.128687] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:21:09.231081] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32217649) [2024-09-11 06:21:09.231982] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5328, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.232202] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035669231226000, start_id=1726035669231226000, end_id=1726035669231226001) [2024-09-11 06:21:09.232652] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.233313] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233341] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233356] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.233367] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.233374] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.233380] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233391] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667297400, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233403] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.233405] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.233411] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.233417] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233429] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035667282422, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.233441] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.233443] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.233446] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.233473] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669233451), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.233481] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.233490] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2411, ddl_arg=NULL) [2024-09-11 06:21:09.335430] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32113308) [2024-09-11 06:21:09.336207] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5329, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.336382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035669335436000, start_id=1726035669335436000, end_id=1726035669335436001) [2024-09-11 06:21:09.336845] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.336860] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.336869] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.337540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337567] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337582] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.337593] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.337599] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.337606] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337629] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.337631] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.337635] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.337639] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337650] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.337660] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.337662] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.337665] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.337686] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669337668), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.337692] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.337700] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.337705] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2277, ddl_arg=NULL) [2024-09-11 06:21:09.439846] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=32008917) [2024-09-11 06:21:09.440644] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5330, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.440833] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035669440727000, start_id=1726035669440727000, end_id=1726035669440727001) [2024-09-11 06:21:09.441238] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.441250] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.441259] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.441917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.441947] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.441963] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.441974] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.441981] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.441986] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.441999] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.442011] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.442013] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.442017] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:09.442022] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.442025] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.442036] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.442047] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.442050] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.442052] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.442077] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669442056), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.442085] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.442093] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.442099] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2255, ddl_arg=NULL) [2024-09-11 06:21:09.544481] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31904255) [2024-09-11 06:21:09.545265] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5331, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.545455] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035669544955000, start_id=1726035669544955000, end_id=1726035669544955001) [2024-09-11 06:21:09.545869] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.545882] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.545891] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.546541] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546569] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546585] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.546599] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.546605] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.546610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546621] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546633] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.546637] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.546640] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:09.546645] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.546649] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546664] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.546675] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.546678] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.546680] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.546703] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669546683), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.546711] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.546717] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.546724] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2245, ddl_arg=NULL) [2024-09-11 06:21:09.649216] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31799519) [2024-09-11 06:21:09.650029] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5332, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.650206] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035669650112000, start_id=1726035669650112000, end_id=1726035669650112001) [2024-09-11 06:21:09.650652] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.650664] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.650672] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.651375] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651404] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651419] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.651431] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.651437] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.651444] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651456] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651467] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.651470] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.651476] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.651485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651496] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.651506] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.651509] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.651512] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.651535] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669651515), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.651543] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.651552] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.651557] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2344, ddl_arg=NULL) [2024-09-11 06:21:09.753880] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31694862) [2024-09-11 06:21:09.754676] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5333, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.754859] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035669754180000, start_id=1726035669754180000, end_id=1726035669754180001) [2024-09-11 06:21:09.755251] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.755264] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.755274] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.755951] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.755982] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.755998] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.756008] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.756014] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.756021] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.756032] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.756043] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.756046] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.756052] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:09.756055] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.756060] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.756071] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.756082] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.756085] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.756107] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669756089), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.756114] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.756121] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.858218] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31590510) [2024-09-11 06:21:09.858982] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5334, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.859173] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035669858359000, start_id=1726035669858359000, end_id=1726035669858359001) [2024-09-11 06:21:09.859587] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:09.859600] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.859611] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.859619] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.860277] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.860307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.860324] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.860336] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.860343] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.860349] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.860359] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.860370] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.860373] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:09.860378] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:09.860385] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:09.860406] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669860389), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.860414] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.860422] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.860426] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2210, ddl_arg=NULL) [2024-09-11 06:21:09.973338] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31475391) [2024-09-11 06:21:09.974059] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5335, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:09.974641] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:09.974658] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:09.974667] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.975296] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975339] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:09.975351] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:09.975356] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:09.975363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975375] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975386] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:09.975388] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.975391] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:09.975394] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:09.975397] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975408] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:09.975417] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:09.975422] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:09.975424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:09.975427] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:09.975446] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035669975430), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:09.975453] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:09.975460] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:09.975464] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2127, ddl_arg=NULL) [2024-09-11 06:21:10.078174] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31370550) [2024-09-11 06:21:10.078984] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5336, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.079165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035670078112000, start_id=1726035670078112000, end_id=1726035670078112001) [2024-09-11 06:21:10.079600] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.079613] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.079622] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.080308] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080338] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080353] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.080365] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.080371] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.080377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080400] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.080403] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.080409] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.080412] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.080417] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080429] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.080438] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.080442] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.080444] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.080447] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.080469] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670080451), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.080477] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.080484] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.080489] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2318, ddl_arg=NULL) [2024-09-11 06:21:10.193350] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31255382) [2024-09-11 06:21:10.194137] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5337, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.194291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035670193713000, start_id=1726035670193713000, end_id=1726035670193713001) [2024-09-11 06:21:10.194699] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.194712] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.194720] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.195360] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195405] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.195420] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.195429] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.195438] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195456] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195469] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.195472] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.195475] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.195481] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.195485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.195508] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.195512] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.195515] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.195518] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.195543] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670195523), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.195551] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.195559] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.195563] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2215, ddl_arg=NULL) [2024-09-11 06:21:10.298072] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31150661) [2024-09-11 06:21:10.298925] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5338, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.299105] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035670298888000, start_id=1726035670298888000, end_id=1726035670298888001) [2024-09-11 06:21:10.299539] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.299553] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:10.300212] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300242] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300257] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.300269] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.300275] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.300282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300294] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300303] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.300306] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.300311] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.300315] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.300319] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300347] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.300360] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.300362] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.300364] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.300367] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.300390] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670300370), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.300398] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.300407] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2338, ddl_arg=NULL) [2024-09-11 06:21:10.402371] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=31046353) [2024-09-11 06:21:10.403160] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5339, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.403326] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035670403055000, start_id=1726035670403055000, end_id=1726035670403055001) [2024-09-11 06:21:10.403787] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:10.404467] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.404496] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.404521] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.404529] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.404553] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670404533), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.404560] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.506977] INFO [RS] add_server (ob_root_service.cpp:7915) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30941762) [2024-09-11 06:21:10.507730] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5340, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.507905] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035670507331000, start_id=1726035670507331000, end_id=1726035670507331001) [2024-09-11 06:21:10.508333] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.508345] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.508360] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.508998] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509042] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.509053] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.509060] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.509065] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509076] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509087] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.509090] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.509094] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.509099] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.509103] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509115] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.509131] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.509134] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.509137] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.509140] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.509159] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670509142), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.509166] INFO [RS] add_server (ob_root_service.cpp:7928) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.509172] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.509178] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55195][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:10.644855] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30803877) [2024-09-11 06:21:10.645664] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5341, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.645846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035670645198000, start_id=1726035670645198000, end_id=1726035670645198001) [2024-09-11 06:21:10.646260] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.646274] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.646282] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.646957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.646987] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.647002] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.647015] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.647021] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.647027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.647038] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.647050] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.647052] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:10.647056] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.647063] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.647086] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670647066), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.647098] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.647106] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.749517] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30699230) [2024-09-11 06:21:10.750314] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5342, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.750490] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035670749625000, start_id=1726035670749625000, end_id=1726035670749625001) [2024-09-11 06:21:10.750939] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.750953] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.750960] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.751622] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751678] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.751691] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.751697] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.751704] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751726] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.751728] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.751735] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.751738] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.751741] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751756] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.751765] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.751767] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.751769] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.751772] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.751808] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670751787), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.751816] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.751824] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.751830] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2315, ddl_arg=NULL) [2024-09-11 06:21:10.854123] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30594602) [2024-09-11 06:21:10.854932] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5343, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.855115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035670854896000, start_id=1726035670854896000, end_id=1726035670854896001) [2024-09-11 06:21:10.855508] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.855522] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.855531] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.856194] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856240] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.856250] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.856256] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.856261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856272] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856284] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.856286] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.856291] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.856295] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.856299] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856311] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.856322] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.856325] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.856328] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.856331] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.856355] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670856334), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.856363] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.856371] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.856379] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2259, ddl_arg=NULL) [2024-09-11 06:21:10.958725] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30490009) [2024-09-11 06:21:10.959525] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5344, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:10.959693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035670959108000, start_id=1726035670959108000, end_id=1726035670959108001) [2024-09-11 06:21:10.960093] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:10.960103] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:10.960110] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:10.960117] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.960791] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960829] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960852] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:10.960867] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:10.960875] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:10.960885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960925] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:10.960930] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.960935] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.960942] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:10.960948] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960967] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:10.960986] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:10.960993] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:10.960998] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:10.961004] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:10.961033] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035670961012), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:10.961043] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:10.961050] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:10.961056] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2332, ddl_arg=NULL) [2024-09-11 06:21:11.063810] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30384946) [2024-09-11 06:21:11.064551] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5345, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.064699] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035671064299000, start_id=1726035671064299000, end_id=1726035671064299001) [2024-09-11 06:21:11.065111] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.065124] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.065133] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.065797] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065825] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669071015, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065841] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.065856] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.065865] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.065874] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065888] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065900] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:11.065903] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.065908] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.065913] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.065917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.065940] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:11.065943] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.065946] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.065948] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:11.065971] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671065952), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.065979] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.065986] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.065992] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2183, ddl_arg=NULL) [2024-09-11 06:21:11.168659] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30280070) [2024-09-11 06:21:11.169456] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5346, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.169631] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035671169519000, start_id=1726035671169519000, end_id=1726035671169519001) [2024-09-11 06:21:11.170102] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.170117] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.170125] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.170804] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170834] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170849] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.170863] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.170869] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.170875] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170896] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:11.170899] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.170902] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.170909] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.170911] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.170932] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:11.170935] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.170938] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.170940] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:11.170963] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671170944), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.170972] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.170980] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.170987] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2330, ddl_arg=NULL) [2024-09-11 06:21:11.273539] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30175191) [2024-09-11 06:21:11.274273] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5347, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.274429] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035671273833000, start_id=1726035671273833000, end_id=1726035671273833001) [2024-09-11 06:21:11.274831] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.274844] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.274851] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.275470] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275499] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275515] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.275526] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.275533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.275538] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275551] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669298010, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275560] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:11.275562] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.275566] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.275569] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.275573] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275582] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035669282747, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8984375GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.275593] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:11.275596] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.275599] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.275601] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:11.275623] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671275605), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.275632] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.275637] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.275641] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2105, ddl_arg=NULL) [2024-09-11 06:21:11.378072] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=30070654) [2024-09-11 06:21:11.378865] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5348, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.379026] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:11.379469] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.379484] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.379492] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.380190] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.380218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.380238] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.380247] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.380267] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671380250), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.380275] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.380283] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.493158] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29955565) [2024-09-11 06:21:11.494024] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5349, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.494209] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035671493653000, start_id=1726035671493653000, end_id=1726035671493653001) [2024-09-11 06:21:11.494676] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.494689] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:11.495341] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495394] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.495407] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.495412] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.495418] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495430] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495442] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:11.495444] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.495448] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.495453] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.495457] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495467] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.495478] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:11.495481] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.495483] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.495485] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:11.495509] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671495488), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.495516] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.495529] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2372, ddl_arg=NULL) [2024-09-11 06:21:11.597659] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29851083) [2024-09-11 06:21:11.598427] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5350, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.598606] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035671597896000, start_id=1726035671597896000, end_id=1726035671597896001) [2024-09-11 06:21:11.599122] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.599135] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.599143] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.599798] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599826] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599842] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.599856] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.599864] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.599872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599895] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:11.599898] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.599901] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.599907] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.599910] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599921] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.599933] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:11.599936] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:11.599939] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.599945] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:11.599972] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671599950), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.599980] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.599987] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.599994] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2337, ddl_arg=NULL) [2024-09-11 06:21:11.712863] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:11.714222] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.714237] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.714247] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.714929] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.714940] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.714966] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671714944), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.714976] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.714985] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.817544] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29631188) [2024-09-11 06:21:11.818379] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5352, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.819007] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.819021] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.819027] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.819714] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.819753] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.819762] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:11.819772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.819806] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671819785), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.819814] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.819824] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.922252] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29526474) [2024-09-11 06:21:11.923020] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5353, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:11.923188] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035671923035000, start_id=1726035671923035000, end_id=1726035671923035001) [2024-09-11 06:21:11.923595] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:11.923606] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:11.923615] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.924261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.924288] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.924303] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:11.924314] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:11.924320] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:11.924325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.924336] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:11.924348] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:11.924352] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.924357] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:11.924379] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035671924363), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:11.924387] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:11.924393] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:11.924398] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2146, ddl_arg=NULL) [2024-09-11 06:21:12.027083] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29421651) [2024-09-11 06:21:12.027870] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5354, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.028033] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035672027359000, start_id=1726035672027359000, end_id=1726035672027359001) [2024-09-11 06:21:12.028478] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:12.028489] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.028497] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.028505] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.029160] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.029188] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.029204] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.029214] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.029220] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.029225] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.029236] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:12.029247] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.029252] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.029275] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672029258), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.029281] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.029287] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.131690] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29317036) [2024-09-11 06:21:12.132488] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5355, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.132704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035672132544000, start_id=1726035672132544000, end_id=1726035672132544001) [2024-09-11 06:21:12.133213] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.133228] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.133236] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.133908] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.133936] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.133952] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.133965] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.133971] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.133984] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.134000] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.134014] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.134018] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.134022] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.134026] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.134031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.134045] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.134060] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.134066] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.134071] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.134076] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.134102] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672134084), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.134111] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.134117] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.134124] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2436, ddl_arg=NULL) [2024-09-11 06:21:12.236557] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29212182) [2024-09-11 06:21:12.237401] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5356, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.237575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035672236760000, start_id=1726035672236760000, end_id=1726035672236760001) [2024-09-11 06:21:12.237990] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.238004] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.238011] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.238662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238695] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238717] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.238734] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.238741] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.238752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238766] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238788] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.238792] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.238797] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.238805] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.238810] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238824] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.238836] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.238839] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.238841] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.238843] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.238864] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672238846), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.238870] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.238878] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.238882] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2327, ddl_arg=NULL) [2024-09-11 06:21:12.341124] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29107607) [2024-09-11 06:21:12.341931] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5357, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.342108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035672342010000, start_id=1726035672342010000, end_id=1726035672342010001) [2024-09-11 06:21:12.342483] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.342496] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.342504] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.343197] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343227] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343243] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.343254] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.343260] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.343266] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343277] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343289] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.343292] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.343298] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.343302] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.343306] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343318] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.343329] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.343332] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.343335] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.343338] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.343363] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672343342), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.343372] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.343378] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.343385] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2264, ddl_arg=NULL) [2024-09-11 06:21:12.445708] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=29003014) [2024-09-11 06:21:12.446497] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5358, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.446668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035672446220000, start_id=1726035672446220000, end_id=1726035672446220001) [2024-09-11 06:21:12.447102] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.447116] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.447124] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.447815] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447850] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447870] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.447881] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.447888] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.447893] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447904] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447919] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.447924] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.447927] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.447932] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.447935] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447949] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.447967] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.447970] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.447972] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.447974] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.447995] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672447978), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.448003] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.448008] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.448015] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2308, ddl_arg=NULL) [2024-09-11 06:21:12.550533] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28898192) [2024-09-11 06:21:12.551326] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5359, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.551502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035672550490000, start_id=1726035672550490000, end_id=1726035672550490001) [2024-09-11 06:21:12.551876] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.551889] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:12.552550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552579] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552595] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.552610] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.552616] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.552623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552647] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.552650] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.552655] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.552658] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.552663] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552673] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.552681] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.552684] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.552711] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672552690), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.552719] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.655256] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28793471) [2024-09-11 06:21:12.656003] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5360, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.656166] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035672655678000, start_id=1726035672655678000, end_id=1726035672655678001) [2024-09-11 06:21:12.656636] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.656654] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.656666] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.657322] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657366] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.657379] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.657386] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.657399] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657411] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657422] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.657425] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.657429] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.657433] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.657437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657448] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.657460] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.657462] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.657465] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.657468] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.657489] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672657472), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.657496] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.657504] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.657508] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2254, ddl_arg=NULL) [2024-09-11 06:21:12.759854] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28688876) [2024-09-11 06:21:12.760619] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5361, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.760771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035672759876000, start_id=1726035672759876000, end_id=1726035672759876001) [2024-09-11 06:21:12.761220] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=42][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.761233] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.761241] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.761882] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.761909] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.761924] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.761936] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.761942] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.761949] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.761961] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.761971] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.761974] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.761980] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.761983] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.761986] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.761996] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.762007] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.762010] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.762012] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.762014] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.762035] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672762017), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.762042] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.762049] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.762053] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2200, ddl_arg=NULL) [2024-09-11 06:21:12.864746] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28583997) [2024-09-11 06:21:12.865533] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5362, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.865705] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035672865143000, start_id=1726035672865143000, end_id=1726035672865143001) [2024-09-11 06:21:12.866121] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.866137] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.866148] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.866872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.866903] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671071406, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.866919] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:12.866932] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:12.866937] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:12.866942] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.866954] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.866965] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:12.866968] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.866972] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.866984] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:12.866987] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.866998] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:12.867009] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:12.867012] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:12.867014] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.867019] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:12.867044] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672867023), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.867052] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.867060] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:12.867066] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2322, ddl_arg=NULL) [2024-09-11 06:21:12.969205] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28479530) [2024-09-11 06:21:12.970014] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5363, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:12.970192] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035672969361000, start_id=1726035672969361000, end_id=1726035672969361001) [2024-09-11 06:21:12.970633] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:12.970645] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:12.970652] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.971509] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:12.971558] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.971570] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:12.971598] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035672971576), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:12.971610] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:12.971617] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.073816] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28374929) [2024-09-11 06:21:13.074552] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5364, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.074710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035673074616000, start_id=1726035673074616000, end_id=1726035673074616001) [2024-09-11 06:21:13.075101] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:13.075110] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.075118] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.075125] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.075745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075808] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075824] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.075837] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.075843] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.075848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075861] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075876] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.075878] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.075882] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.075885] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.075890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075901] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.075912] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.075914] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.075916] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.075919] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.075938] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673075922), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.075946] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.075952] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.075960] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2145, ddl_arg=NULL) [2024-09-11 06:21:13.182973] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28265755) [2024-09-11 06:21:13.183755] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5365, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.183933] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035673183781000, start_id=1726035673183781000, end_id=1726035673183781001) [2024-09-11 06:21:13.183998] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=46, avg_log_batch_cnt=1, total_group_log_size=35447, avg_group_log_size=770) [2024-09-11 06:21:13.184011] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=35257) [2024-09-11 06:21:13.184344] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.184355] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.184363] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.185021] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185049] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185065] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.185075] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.185081] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.185086] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185109] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.185113] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.185117] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.185122] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.185126] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185137] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671283079, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.185148] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.185151] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.185154] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.185159] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.185188] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673185165), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.185196] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.185203] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.185209] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2238, ddl_arg=NULL) [2024-09-11 06:21:13.287553] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28161176) [2024-09-11 06:21:13.288430] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5366, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.288606] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035673287949000, start_id=1726035673287949000, end_id=1726035673287949001) [2024-09-11 06:21:13.289025] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.289039] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.289048] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.289727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289780] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.289796] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.289804] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.289811] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035671298493, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7519531GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289834] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.289837] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.289841] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.289846] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.289852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289864] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.289875] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.289878] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.289880] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.289882] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.289908] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673289886), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.289916] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.289928] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.289933] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2382, ddl_arg=NULL) [2024-09-11 06:21:13.402488] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=28046236) [2024-09-11 06:21:13.403227] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5367, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.403394] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035673402730000, start_id=1726035673402730000, end_id=1726035673402730001) [2024-09-11 06:21:13.403826] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.403838] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.403844] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.404482] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404510] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404525] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.404536] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.404543] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.404550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404562] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404571] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.404573] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.404576] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.404579] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.404582] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404593] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.404603] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.404605] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.404607] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.404610] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.404630] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673404613), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.404637] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.404644] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.404649] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:13.507239] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27941500) [2024-09-11 06:21:13.508024] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5368, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.508212] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035673508121000, start_id=1726035673508121000, end_id=1726035673508121001) [2024-09-11 06:21:13.508645] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.508658] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.508664] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.509317] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509349] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509367] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.509376] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.509382] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.509389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509402] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509411] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.509413] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.509416] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.509419] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.509423] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.509454] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.509457] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.509460] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.509464] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.509488] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673509468), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.509498] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.509506] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.509513] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2276, ddl_arg=NULL) [2024-09-11 06:21:13.611752] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27836979) [2024-09-11 06:21:13.612650] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5369, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.612899] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035673612136000, start_id=1726035673612136000, end_id=1726035673612136001) [2024-09-11 06:21:13.613381] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.613396] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.613404] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.614107] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614143] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614166] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.614185] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.614194] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.614204] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614225] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614244] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.614250] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.614256] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.614262] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.614271] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614297] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.614317] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.614321] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.614328] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.614334] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.614368] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673614342), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.614380] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.614391] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.614401] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [DDL] execute ddl like stmt(ret=-4122, cost=2651, ddl_arg=NULL) [2024-09-11 06:21:13.716226] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27732503) [2024-09-11 06:21:13.717058] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5370, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.717232] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035673716393000, start_id=1726035673716393000, end_id=1726035673716393001) [2024-09-11 06:21:13.717619] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.717631] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.717639] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.718293] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718323] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718339] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.718351] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.718357] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.718364] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718376] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718387] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.718390] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.718394] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.718399] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.718402] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718413] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.718423] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.718425] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.718427] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.718432] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.718455] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673718437), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.718464] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.718471] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.718475] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2251, ddl_arg=NULL) [2024-09-11 06:21:13.821393] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27627338) [2024-09-11 06:21:13.822207] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5371, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.822408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035673821554000, start_id=1726035673821554000, end_id=1726035673821554001) [2024-09-11 06:21:13.822850] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.822864] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.822869] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.823550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823597] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.823609] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.823615] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.823621] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823633] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823644] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.823647] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.823650] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.823654] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:13.823659] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823670] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.823681] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:13.823683] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:13.823685] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.823688] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:13.823708] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673823691), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.823713] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.823719] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:13.823725] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2334, ddl_arg=NULL) [2024-09-11 06:21:13.925937] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27522793) [2024-09-11 06:21:13.926710] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5372, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:13.926874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035673925889000, start_id=1726035673925889000, end_id=1726035673925889001) [2024-09-11 06:21:13.927359] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:13.927375] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:13.927381] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.928051] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.928077] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:13.928091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:13.928104] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:13.928111] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:13.928116] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:13.928130] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:13.928133] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.928138] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:13.928171] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035673928144), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:13.928178] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:13.928184] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.030414] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27418312) [2024-09-11 06:21:14.031214] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5373, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.031377] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035674030954000, start_id=1726035674030954000, end_id=1726035674030954001) [2024-09-11 06:21:14.031815] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.031828] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.031836] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.032504] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032534] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032550] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.032561] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.032567] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.032574] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032586] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032597] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.032599] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.032603] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.032608] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.032612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032628] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.032639] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.032641] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.032644] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.032646] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.032670] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674032650), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.032678] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.032685] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.032690] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:21:14.139550] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27309180) [2024-09-11 06:21:14.140322] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5374, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.140501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035674139494000, start_id=1726035674139494000, end_id=1726035674139494001) [2024-09-11 06:21:14.140968] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:14.140976] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.140984] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.140992] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.141610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141637] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141653] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.141666] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.141672] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.141677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141688] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141700] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.141703] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.141708] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.141714] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.141717] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141729] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.141740] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.141742] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.141746] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.141748] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.141769] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674141753), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.141787] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.141792] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.141796] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2247, ddl_arg=NULL) [2024-09-11 06:21:14.244191] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27204547) [2024-09-11 06:21:14.245020] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5375, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.245201] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035674244760000, start_id=1726035674244760000, end_id=1726035674244760001) [2024-09-11 06:21:14.245257] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28261, avg_group_log_size=807) [2024-09-11 06:21:14.245267] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28261) [2024-09-11 06:21:14.245624] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.245637] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.245645] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.246313] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246358] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.246375] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.246381] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.246385] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246396] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246408] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.246410] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.246413] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.246416] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.246419] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246430] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.246439] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.246442] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.246444] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.246446] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.246469] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674246450), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.246476] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.246483] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.246490] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2301, ddl_arg=NULL) [2024-09-11 06:21:14.348708] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=27100023) [2024-09-11 06:21:14.349530] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5376, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.349699] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035674348837000, start_id=1726035674348837000, end_id=1726035674348837001) [2024-09-11 06:21:14.350102] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.350117] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.350125] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.350786] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350815] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350831] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.350843] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.350847] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.350852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350863] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350875] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.350878] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.350885] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.350889] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.350894] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.350916] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.350919] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.350922] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.350927] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.350949] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674350930), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.350957] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.350965] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.350971] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2266, ddl_arg=NULL) [2024-09-11 06:21:14.463923] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26984815) [2024-09-11 06:21:14.464708] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5377, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.464894] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035674464514000, start_id=1726035674464514000, end_id=1726035674464514001) [2024-09-11 06:21:14.465248] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.465263] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.465275] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.465968] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.465997] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.466012] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.466023] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.466029] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.466034] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.466045] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.466055] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.466058] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.466062] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.466066] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.466074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.466092] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.466105] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.466108] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.466110] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.466113] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.466133] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674466116), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.466142] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.466147] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.466152] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:14.568480] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26880268) [2024-09-11 06:21:14.569242] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5378, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.569408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035674568708000, start_id=1726035674568708000, end_id=1726035674568708001) [2024-09-11 06:21:14.569900] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.569911] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.569919] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.570554] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570588] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570603] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.570613] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.570616] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.570623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570651] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.570654] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.570657] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.570660] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.570665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570676] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.570686] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.570689] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.570691] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.570693] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.570715] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674570699), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.570722] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.570728] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.570734] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2254, ddl_arg=NULL) [2024-09-11 06:21:14.683219] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26765535) [2024-09-11 06:21:14.684065] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5379, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.684230] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035674683321000, start_id=1726035674683321000, end_id=1726035674683321001) [2024-09-11 06:21:14.684710] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.684726] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:14.685474] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.685505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.685519] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.685532] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.685539] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.685566] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674685542), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.685574] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.787572] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26661150) [2024-09-11 06:21:14.788388] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5380, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.788562] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035674788534000, start_id=1726035674788534000, end_id=1726035674788534001) [2024-09-11 06:21:14.789013] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.789027] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.789037] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.789669] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789696] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789712] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.789723] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.789730] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.789735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789747] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.789758] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.789761] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.789764] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.789767] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789784] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.789796] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.789799] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.789802] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.789804] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.789829] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674789808), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.789837] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.789843] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.789847] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:21:14.892272] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26556466) [2024-09-11 06:21:14.893104] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:14.893678] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.893692] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.893700] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.894437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.894466] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.894481] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.894496] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.894504] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.894527] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674894507), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.894535] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.894542] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.996665] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26452060) [2024-09-11 06:21:14.997427] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5382, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:14.997578] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035674996715000, start_id=1726035674996715000, end_id=1726035674996715001) [2024-09-11 06:21:14.998002] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:14.998017] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:14.998023] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.998652] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998685] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673071772, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998701] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:14.998715] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:14.998722] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:14.998727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998739] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:14.998759] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.998762] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.998765] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:14.998768] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998785] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:14.998794] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:14.998796] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:14.998799] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:14.998801] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:14.998823] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035674998804), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:14.998829] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:14.998834] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:14.998838] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2174, ddl_arg=NULL) [2024-09-11 06:21:15.101125] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26347600) [2024-09-11 06:21:15.101936] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5383, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.102138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035675101866000, start_id=1726035675101866000, end_id=1726035675101866001) [2024-09-11 06:21:15.102597] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.102611] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.102620] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.103282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103311] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103326] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.103339] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.103351] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:15.103356] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103370] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103381] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:15.103384] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.103389] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.103392] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:15.103395] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103405] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673283468, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.103415] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:15.103418] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.103421] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.103425] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:15.103445] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675103428), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.103454] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.103462] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.103466] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2343, ddl_arg=NULL) [2024-09-11 06:21:15.206707] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26242014) [2024-09-11 06:21:15.207444] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5384, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.207610] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035675207118000, start_id=1726035675207118000, end_id=1726035675207118001) [2024-09-11 06:21:15.208054] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:15.208065] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.208072] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.208080] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.208929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.208965] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.208987] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.209002] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.209010] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:15.209018] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.209037] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035673298980, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.209055] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:15.209059] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.209064] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.209072] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:15.209081] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.209112] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675209090), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.209122] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.209132] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.311035] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26137699) [2024-09-11 06:21:15.311890] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5385, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.312093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035675311177000, start_id=1726035675311177000, end_id=1726035675311177001) [2024-09-11 06:21:15.312169] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=37, avg_log_batch_cnt=1, total_group_log_size=28927, avg_group_log_size=781) [2024-09-11 06:21:15.312188] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28927) [2024-09-11 06:21:15.312510] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.312523] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.312530] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.313175] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313203] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313218] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.313230] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.313236] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:15.313240] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313263] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:15.313265] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.313268] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.313271] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:15.313274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.313294] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:15.313296] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.313299] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.313301] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:15.313324] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675313306), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.313330] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.313337] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.313340] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2307, ddl_arg=NULL) [2024-09-11 06:21:15.415435] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=26033296) [2024-09-11 06:21:15.416195] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5386, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.416373] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035675415360000, start_id=1726035675415360000, end_id=1726035675415360001) [2024-09-11 06:21:15.416808] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.416820] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.416827] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.417464] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:15.417501] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.417510] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.417533] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675417514), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.417545] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.417550] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.519870] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25928883) [2024-09-11 06:21:15.520716] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5387, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.520976] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035675520616000, start_id=1726035675520616000, end_id=1726035675520616001) [2024-09-11 06:21:15.521454] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.521473] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.521482] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.522221] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:15.522264] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.522275] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.522280] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.522308] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675522284), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.522316] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.522322] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.624738] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25823998) [2024-09-11 06:21:15.625529] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5388, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.625710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035675624778000, start_id=1726035675624778000, end_id=1726035675624778001) [2024-09-11 06:21:15.626170] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.626182] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.626190] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.626852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626882] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.626908] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.626914] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:15.626921] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626945] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:15.626948] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.626950] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.626954] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:15.626956] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626967] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.626978] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:15.626980] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.626983] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.626985] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:15.627006] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675626988), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.627013] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.627019] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.627024] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2288, ddl_arg=NULL) [2024-09-11 06:21:15.810669] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.810687] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.810697] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.811407] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.811438] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.811453] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.811466] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.811473] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.811498] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675811477), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.811507] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.811514] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.929312] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25519434) [2024-09-11 06:21:15.930128] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5390, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:15.930318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035675929834000, start_id=1726035675929834000, end_id=1726035675929834001) [2024-09-11 06:21:15.930770] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:15.930814] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:15.930820] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.931497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931526] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931541] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:15.931556] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:15.931562] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:15.931568] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931579] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931590] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:15.931592] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.931599] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.931603] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:15.931606] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:15.931629] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:15.931632] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:15.931634] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:15.931637] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:15.931663] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035675931641), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:15.931672] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:15.931679] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:15.931684] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:16.035418] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5391, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.035639] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035676034958000, start_id=1726035676034958000, end_id=1726035676034958001) [2024-09-11 06:21:16.036069] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.036083] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.036093] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.036816] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.036847] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.036863] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.036874] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.036880] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.036887] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.036894] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.036916] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676036898), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.036924] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.036932] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.139605] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25309146) [2024-09-11 06:21:16.140481] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5392, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.140697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035676140121000, start_id=1726035676140121000, end_id=1726035676140121001) [2024-09-11 06:21:16.141160] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.141175] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.141185] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.141862] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141892] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141909] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.141921] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.141927] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.141934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141945] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141954] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:16.141959] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.141963] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.141968] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:16.141971] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141982] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.141993] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:16.141996] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.141998] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.142000] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:16.142025] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676142004), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.142032] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.142040] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.244967] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25203757) [2024-09-11 06:21:16.245757] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:16.246408] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:16.246420] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.246430] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.246438] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.247328] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.247366] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.247400] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.247411] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.247437] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676247415), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.247450] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.247458] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.349977] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=25098752) [2024-09-11 06:21:16.350827] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5394, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.351008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035676350509000, start_id=1726035676350509000, end_id=1726035676350509001) [2024-09-11 06:21:16.351407] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.351421] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.351430] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.352157] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.352186] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.352209] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.352223] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.352230] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.352236] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.352250] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.352255] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.352278] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676352258), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.352286] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.352292] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.455011] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24993719) [2024-09-11 06:21:16.455820] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5395, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.456003] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035676455690000, start_id=1726035676455690000, end_id=1726035676455690001) [2024-09-11 06:21:16.456438] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.456452] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.456459] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.457119] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.457149] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.457162] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.457175] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.457182] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:16.457188] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.457196] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.457224] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676457200), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.457232] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.457238] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.560208] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24888519) [2024-09-11 06:21:16.560993] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5396, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.561178] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035676560431000, start_id=1726035676560431000, end_id=1726035676560431001) [2024-09-11 06:21:16.561647] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.561663] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.561676] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.562389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562419] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562436] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.562449] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.562456] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.562462] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562473] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562486] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:16.562490] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.562494] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.562500] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:16.562503] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562515] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.562528] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:16.562531] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.562533] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.562536] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:16.562558] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676562540), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.562563] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.562571] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.666510] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.666525] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.666534] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.667217] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.667229] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.667250] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676667233), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.667260] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.667267] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.770564] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24678163) [2024-09-11 06:21:16.771338] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5398, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.771525] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035676770832000, start_id=1726035676770832000, end_id=1726035676770832001) [2024-09-11 06:21:16.771959] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.771973] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.771982] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.772641] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772670] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772685] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.772696] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.772701] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.772711] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772742] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:16.772745] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.772748] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.772752] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:16.772757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772769] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.772786] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:16.772790] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.772817] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676772796), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.772828] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.772833] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.875628] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24573104) [2024-09-11 06:21:16.876473] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5399, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.876680] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035676876057000, start_id=1726035676876057000, end_id=1726035676876057001) [2024-09-11 06:21:16.877103] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:16.877117] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:16.877125] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.877794] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.877825] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.877841] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.877851] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.877857] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.877862] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.877874] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.877886] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:16.877889] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.877893] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.877899] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.877924] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676877903), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.877933] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.877943] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:16.980740] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24467985) [2024-09-11 06:21:16.981522] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5400, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:16.981693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035676981126000, start_id=1726035676981126000, end_id=1726035676981126001) [2024-09-11 06:21:16.982119] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:16.982821] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675072132, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7207031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982867] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:16.982877] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:16.982883] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:16.982890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982905] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982921] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:16.982925] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.982931] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.982935] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:16.982939] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982952] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:16.982966] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:16.982969] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:16.982971] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:16.982973] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:16.982995] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035676982976), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:16.983004] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:16.983011] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2273, ddl_arg=NULL) [2024-09-11 06:21:17.085716] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24363019) [2024-09-11 06:21:17.086480] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5401, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.086644] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035677086434000, start_id=1726035677086434000, end_id=1726035677086434001) [2024-09-11 06:21:17.087114] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.087127] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.087136] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.087843] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087887] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.087898] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.087904] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.087909] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087920] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087929] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.087931] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.087937] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.087940] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.087945] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675283877, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.087966] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.087968] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.087970] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.087972] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:17.087993] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677087975), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.088000] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.088007] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.088013] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2299, ddl_arg=NULL) [2024-09-11 06:21:17.190594] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24258149) [2024-09-11 06:21:17.191386] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5402, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.191556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035677191501000, start_id=1726035677191501000, end_id=1726035677191501001) [2024-09-11 06:21:17.192032] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.192048] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.192059] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.192760] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.192826] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=65] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.192841] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.192852] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.192858] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:17.192864] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.192876] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.192898] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677192879), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.192906] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.192913] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.295361] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24153367) [2024-09-11 06:21:17.296218] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5403, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.296389] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035677295762000, start_id=1726035677295762000, end_id=1726035677295762001) [2024-09-11 06:21:17.296812] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:17.296821] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.296831] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.296838] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.297488] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297518] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297533] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.297544] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.297551] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.297556] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297567] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035675299434, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7617188GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297578] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.297582] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.297586] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.297589] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.297592] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297604] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.297615] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.297620] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.297625] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.297631] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:17.297656] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677297635), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.297665] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.297671] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.297675] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2317, ddl_arg=NULL) [2024-09-11 06:21:17.399796] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=24048950) [2024-09-11 06:21:17.400533] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5404, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.400696] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035677399683000, start_id=1726035677399683000, end_id=1726035677399683001) [2024-09-11 06:21:17.400754] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=43, avg_log_batch_cnt=1, total_group_log_size=35427, avg_group_log_size=823) [2024-09-11 06:21:17.400764] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=35238) [2024-09-11 06:21:17.401156] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.401170] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.401178] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.401825] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.401853] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:17.401870] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.401883] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.401891] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.401913] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677401894), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.401921] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.401927] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.504270] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23944465) [2024-09-11 06:21:17.505084] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5405, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.505271] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035677504954000, start_id=1726035677504954000, end_id=1726035677504954001) [2024-09-11 06:21:17.505703] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.505716] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.505726] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.506399] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506428] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506447] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.506459] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.506465] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.506473] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506488] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506502] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.506511] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.506516] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.506521] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.506528] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506544] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.506560] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.506567] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.506573] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.506579] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:17.506608] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677506587), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.506615] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.506624] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.506631] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2363, ddl_arg=NULL) [2024-09-11 06:21:17.608615] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23840115) [2024-09-11 06:21:17.609396] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5406, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.609577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035677609165000, start_id=1726035677609165000, end_id=1726035677609165001) [2024-09-11 06:21:17.610018] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.610031] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.610039] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.610675] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610708] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610726] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.610737] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.610742] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.610746] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610767] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.610769] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.610772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.610783] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.610786] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.610807] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.610809] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.610811] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.610813] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:17.610833] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677610816), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.610844] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.610853] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.610858] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2244, ddl_arg=NULL) [2024-09-11 06:21:17.723715] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23725003) [2024-09-11 06:21:17.724478] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5407, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.724648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035677723914000, start_id=1726035677723914000, end_id=1726035677723914001) [2024-09-11 06:21:17.725037] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.725050] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.725056] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.725699] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725726] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725742] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.725752] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.725758] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.725763] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725805] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=42] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725817] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.725820] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.725826] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.725829] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.725832] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725843] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.725853] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.725855] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.725859] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.725861] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:17.725884] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677725866), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.725892] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.725897] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.828654] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23620080) [2024-09-11 06:21:17.829434] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5408, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.829632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035677829135000, start_id=1726035677829135000, end_id=1726035677829135001) [2024-09-11 06:21:17.830079] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.830100] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.830110] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.830965] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.830992] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.831007] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:17.831019] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:17.831026] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:17.831035] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.831047] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.831056] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:17.831059] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.831063] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.831066] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:17.831072] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.831083] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.831094] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:17.831098] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:17.831105] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.831108] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:17.831132] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677831113), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.831139] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.831146] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:17.831150] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2498, ddl_arg=NULL) [2024-09-11 06:21:17.943772] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23504967) [2024-09-11 06:21:17.944513] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5409, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:17.944666] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035677943891000, start_id=1726035677943891001, end_id=1726035677943891002) [2024-09-11 06:21:17.944990] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:17.945002] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:17.945010] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.945752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.945797] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=45] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:17.945824] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.945833] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:17.945859] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035677945841), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:17.945869] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:17.945879] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.048263] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23400465) [2024-09-11 06:21:18.049020] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5410, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.049172] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035678048891000, start_id=1726035678048891000, end_id=1726035678048891001) [2024-09-11 06:21:18.049576] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.049587] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.049595] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.050248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050277] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050291] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.050303] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.050308] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.050314] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050337] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.050340] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.050344] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.050347] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.050350] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050362] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.050373] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.050375] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.050377] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.050380] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:18.050399] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678050383), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.050406] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.050412] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.050418] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2155, ddl_arg=NULL) [2024-09-11 06:21:18.152667] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23296057) [2024-09-11 06:21:18.153470] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5411, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.153632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1726035678152942000, start_id=1726035678152942000, end_id=1726035678152942001) [2024-09-11 06:21:18.154059] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.154073] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.154081] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.154766] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154801] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154815] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.154827] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.154833] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.154840] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154854] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154865] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.154868] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.154871] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.154874] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.154877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154887] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.154899] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.154901] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.154903] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.154905] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:18.154927] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678154909), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.154935] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.154942] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.154948] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2284, ddl_arg=NULL) [2024-09-11 06:21:18.258062] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23190676) [2024-09-11 06:21:18.258855] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5412, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.259029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035678258116000, start_id=1726035678258116000, end_id=1726035678258116001) [2024-09-11 06:21:18.259445] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.259459] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.259467] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.260150] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260179] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260193] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.260204] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.260211] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.260218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260229] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260241] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.260243] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.260247] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.260253] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.260256] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260266] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.260277] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.260280] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.260282] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.260284] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:18.260308] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678260288), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.260317] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.260325] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.260329] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2269, ddl_arg=NULL) [2024-09-11 06:21:18.372850] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=23075894) [2024-09-11 06:21:18.373559] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5413, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.373722] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035678373030000, start_id=1726035678373030000, end_id=1726035678373030001) [2024-09-11 06:21:18.374240] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:18.374251] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.374259] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.374266] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.375002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375047] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.375057] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.375063] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.375069] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375080] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375092] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.375095] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.375099] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.375103] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.375109] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375121] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.375130] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.375133] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.375135] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.375137] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:18.375162] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678375141), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.375170] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.375178] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.375183] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2335, ddl_arg=NULL) [2024-09-11 06:21:18.477542] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:18.478954] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.478969] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.478978] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.479631] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.479675] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.479682] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.479707] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678479686), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.479718] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.479725] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.582094] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22866630) [2024-09-11 06:21:18.582898] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5415, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.583089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035678582387000, start_id=1726035678582387000, end_id=1726035678582387001) [2024-09-11 06:21:18.583533] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.583547] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.583556] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.584280] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.584310] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:18.584334] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.584347] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.584374] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678584352), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.584382] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.584391] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.686931] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22761793) [2024-09-11 06:21:18.687744] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5416, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.687942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035678687666000, start_id=1726035678687666000, end_id=1726035678687666001) [2024-09-11 06:21:18.688396] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.688409] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.688417] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.689088] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.689118] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.689134] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.689148] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.689155] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.689184] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678689158), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.689192] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.689197] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.791378] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22657341) [2024-09-11 06:21:18.792151] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5417, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.792334] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035678791781000, start_id=1726035678791781000, end_id=1726035678791781001) [2024-09-11 06:21:18.792839] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.792852] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.792860] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.793498] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793532] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793548] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.793559] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.793565] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.793571] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793581] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793592] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.793595] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.793601] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.793604] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.793607] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.793627] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.793630] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.793632] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.793634] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:18.793660] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678793641), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.793667] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.793674] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:18.793680] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2304, ddl_arg=NULL) [2024-09-11 06:21:18.895864] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22552862) [2024-09-11 06:21:18.896610] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5418, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:18.896843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035678896819000, start_id=1726035678896819000, end_id=1726035678896819001) [2024-09-11 06:21:18.897318] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:18.897331] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:18.897340] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.897999] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898029] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898045] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:18.898055] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:18.898062] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:18.898067] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:18.898094] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.898100] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.898103] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:18.898109] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898119] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:18.898130] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:18.898133] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:18.898135] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:18.898137] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:18.898159] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035678898143), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:18.898167] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:18.898172] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.000373] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22448355) [2024-09-11 06:21:19.001134] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5419, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.001292] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035679001053000, start_id=1726035679001053000, end_id=1726035679001053001) [2024-09-11 06:21:19.001746] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.001760] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.001771] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.002427] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002454] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677072463, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002469] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.002480] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.002487] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.002499] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002513] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002525] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.002528] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.002531] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.002534] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.002537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.002560] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.002563] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.002566] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.002568] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.002637] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679002572), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.002645] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.002650] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.002657] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2286, ddl_arg=NULL) [2024-09-11 06:21:19.104914] INFO [RS] add_server (ob_root_service.cpp:7915) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22343817) [2024-09-11 06:21:19.105716] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5420, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.105916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035679105181000, start_id=1726035679105181000, end_id=1726035679105181001) [2024-09-11 06:21:19.106355] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.106368] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:19.107026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107071] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.107083] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.107088] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.107095] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107107] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677299865, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8125GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107118] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.107120] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.107123] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.107127] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.107132] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107142] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035677284299, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.107153] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.107155] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.107157] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.107166] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.107187] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679107169), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.107194] INFO [RS] add_server (ob_root_service.cpp:7928) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.107203] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55208][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2292, ddl_arg=NULL) [2024-09-11 06:21:19.209508] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22239229) [2024-09-11 06:21:19.210599] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:19.212241] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.212257] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.212289] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679212264), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.212302] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.314195] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22134535) [2024-09-11 06:21:19.315034] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5422, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.315243] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035679314595000, start_id=1726035679314595000, end_id=1726035679314595001) [2024-09-11 06:21:19.315644] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.315657] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.315663] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.316326] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316358] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316374] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.316386] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.316392] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.316398] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316410] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316422] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.316424] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.316430] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.316433] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.316439] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.316460] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.316463] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.316465] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.316467] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.316498] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679316470), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.316508] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.316515] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.316521] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2328, ddl_arg=NULL) [2024-09-11 06:21:19.418474] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=22030253) [2024-09-11 06:21:19.419284] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5423, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.419451] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035679418534000, start_id=1726035679418534000, end_id=1726035679418534001) [2024-09-11 06:21:19.419877] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:19.419887] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.419894] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.419901] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.420547] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420574] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420589] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.420599] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.420605] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.420612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420624] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420635] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.420638] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.420644] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.420647] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.420650] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.420672] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.420675] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.420677] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.420680] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.420700] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679420683), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.420707] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.420713] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.420717] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2244, ddl_arg=NULL) [2024-09-11 06:21:19.534055] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21914679) [2024-09-11 06:21:19.534902] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5424, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.535090] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035679534293000, start_id=1726035679534293000, end_id=1726035679534293001) [2024-09-11 06:21:19.535209] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=68] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=28254, avg_group_log_size=807) [2024-09-11 06:21:19.535224] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28253) [2024-09-11 06:21:19.535580] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.535591] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.535601] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.536319] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536359] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536374] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.536386] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.536392] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.536399] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536410] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536421] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.536424] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.536430] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.536433] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.536439] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.536460] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.536464] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.536466] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.536468] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.536491] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679536471), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.536502] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.536507] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.536512] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2460, ddl_arg=NULL) [2024-09-11 06:21:19.638501] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21810224) [2024-09-11 06:21:19.639221] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5425, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.639379] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035679638496000, start_id=1726035679638496000, end_id=1726035679638496001) [2024-09-11 06:21:19.639856] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.639870] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.639877] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.640518] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640564] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.640576] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.640583] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.640609] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640623] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640635] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.640637] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.640641] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.640645] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.640649] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640660] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.640670] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.640673] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.640677] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.640682] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.640704] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679640686), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.640712] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.640718] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.640723] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2224, ddl_arg=NULL) [2024-09-11 06:21:19.753307] INFO [RS] add_server (ob_root_service.cpp:7915) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21695424) [2024-09-11 06:21:19.754059] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5426, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.754234] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035679753386000, start_id=1726035679753386000, end_id=1726035679753386001) [2024-09-11 06:21:19.754664] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.754676] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.754683] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.755377] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:19.755413] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.755424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.755446] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679755427), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.755455] INFO [RS] add_server (ob_root_service.cpp:7928) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.755461] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.857974] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21590754) [2024-09-11 06:21:19.858754] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5427, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.858925] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035679858589000, start_id=1726035679858589000, end_id=1726035679858589001) [2024-09-11 06:21:19.859356] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.859366] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.859375] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.860058] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860088] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860101] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.860111] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.860117] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.860122] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860133] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860145] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.860148] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.860152] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.860155] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.860158] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860167] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.860177] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.860179] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.860182] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.860184] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.860208] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679860188), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.860215] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.860221] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.860225] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2252, ddl_arg=NULL) [2024-09-11 06:21:19.962389] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21486336) [2024-09-11 06:21:19.963214] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5428, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:19.963409] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035679962836000, start_id=1726035679962836000, end_id=1726035679962836001) [2024-09-11 06:21:19.963855] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:19.963870] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:19.963879] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.964573] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964605] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964619] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:19.964633] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:19.964640] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:19.964650] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964667] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964682] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:19.964686] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.964690] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.964694] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:19.964697] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964708] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:19.964721] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:19.964725] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:19.964730] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:19.964733] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:19.964763] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035679964741), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:19.964772] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:19.964788] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:19.964794] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2406, ddl_arg=NULL) [2024-09-11 06:21:20.066855] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21381868) [2024-09-11 06:21:20.067627] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5429, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.067813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680067348000, start_id=1726035680067348000, end_id=1726035680067348001) [2024-09-11 06:21:20.068171] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.068183] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.068192] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.068852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.068882] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.068897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.068908] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.068915] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.068920] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.068931] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.068953] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:20.068957] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.068960] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.068964] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:20.068971] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.068983] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.069002] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:20.069006] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.069010] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.069013] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:20.069036] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680069019), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.069044] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.069059] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.069063] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2210, ddl_arg=NULL) [2024-09-11 06:21:20.171305] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21277421) [2024-09-11 06:21:20.172145] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5430, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.172339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680171524000, start_id=1726035680171524000, end_id=1726035680171524001) [2024-09-11 06:21:20.172782] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.172799] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.172810] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.173560] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173591] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173606] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.173618] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.173624] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.173630] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173643] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173653] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:20.173656] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.173662] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.173665] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:20.173670] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173681] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.173692] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:20.173694] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.173697] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.173700] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:20.173723] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680173703), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.173732] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.173739] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.173744] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2441, ddl_arg=NULL) [2024-09-11 06:21:20.276209] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21172518) [2024-09-11 06:21:20.277154] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:20.277356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035680276834000, start_id=1726035680276834000, end_id=1726035680276834001) [2024-09-11 06:21:20.277817] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.278660] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.278695] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.278720] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.278737] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.278746] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.278772] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680278750), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.278789] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.380589] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=21068127) [2024-09-11 06:21:20.381292] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5432, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.381443] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680381056000, start_id=1726035680381056000, end_id=1726035680381056001) [2024-09-11 06:21:20.381859] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.381873] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.381882] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.382527] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382556] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382572] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.382582] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.382587] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.382595] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382608] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382620] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:20.382623] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.382628] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.382632] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:20.382638] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382656] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.382668] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:20.382671] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.382674] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.382676] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:20.382698] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680382680), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.382706] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.382711] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.382716] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2129, ddl_arg=NULL) [2024-09-11 06:21:20.485225] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20963499) [2024-09-11 06:21:20.486055] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5433, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.486235] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680485281000, start_id=1726035680485281000, end_id=1726035680485281001) [2024-09-11 06:21:20.486643] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:20.486654] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.486662] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.486668] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.487374] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.487385] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.487409] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680487390), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.487415] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.487423] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.487427] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2203, ddl_arg=NULL) [2024-09-11 06:21:20.589740] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:20.590556] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5434, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.590713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035680590558000, start_id=1726035680590558000, end_id=1726035680590558001) [2024-09-11 06:21:20.591150] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.591164] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.591174] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.591824] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.591851] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.591867] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.591876] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.591880] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.591886] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.591898] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.591906] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.591927] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680591909), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.591935] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.591942] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.696202] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20752520) [2024-09-11 06:21:20.697036] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5435, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.697227] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680696724000, start_id=1726035680696724000, end_id=1726035680696724001) [2024-09-11 06:21:20.697647] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.697661] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.697670] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.698366] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698395] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698410] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.698424] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.698430] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.698436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698446] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698457] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:20.698460] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.698465] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.698471] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:20.698474] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698486] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.698497] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:20.698500] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.698502] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.698505] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:20.698532] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680698512), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.698541] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.698546] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.698550] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2351, ddl_arg=NULL) [2024-09-11 06:21:20.800750] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:20.801661] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1726035680800836000, start_id=1726035680800836000, end_id=1726035680800836001) [2024-09-11 06:21:20.802100] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.802114] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.802124] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.802745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.802789] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=43] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.802810] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.802819] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.802826] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.802831] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.802842] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.802848] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.802874] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680802852), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.802882] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.802887] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:20.905226] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20543497) [2024-09-11 06:21:20.906014] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5437, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:20.906200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035680905276000, start_id=1726035680905276000, end_id=1726035680905276001) [2024-09-11 06:21:20.906626] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:20.906640] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:20.906648] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.907300] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907329] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907343] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:20.907356] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:20.907362] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:20.907367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907391] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:20.907394] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:20.907398] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.907401] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:20.907407] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907417] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:20.907428] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:20.907432] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:20.907435] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:20.907459] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035680907439), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:20.907468] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:20.907474] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.009842] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20438896) [2024-09-11 06:21:21.010583] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5438, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.010757] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035681010513000, start_id=1726035681010513000, end_id=1726035681010513001) [2024-09-11 06:21:21.011247] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.011260] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.011268] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.011976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012004] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679072810, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012017] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.012027] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.012032] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.012039] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012052] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012063] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.012065] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.012068] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.012071] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.012074] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.012102] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.012104] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.012106] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.012111] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.012132] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681012115), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.012140] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.012148] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.012151] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2311, ddl_arg=NULL) [2024-09-11 06:21:21.114542] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20334179) [2024-09-11 06:21:21.115292] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5439, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.115456] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035681114665000, start_id=1726035681114665000, end_id=1726035681114665001) [2024-09-11 06:21:21.115897] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.115911] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.115918] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.116530] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116563] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116580] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.116592] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.116598] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.116615] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116630] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116642] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.116645] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.116649] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.116651] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.116656] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.116677] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.116680] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.116683] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.116685] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.116707] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681116688), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.116716] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.116721] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.116726] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2186, ddl_arg=NULL) [2024-09-11 06:21:21.219046] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20229696) [2024-09-11 06:21:21.219762] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5440, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.219974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1726035681219893000, start_id=1726035681219893000, end_id=1726035681219893001) [2024-09-11 06:21:21.220431] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.220445] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.220460] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.221139] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221166] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221183] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.221194] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.221200] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.221205] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679300323, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7949219GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221229] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.221234] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.221238] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.221242] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.221246] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221275] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035679284619, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.221285] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.221288] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.221290] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.221292] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.221311] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681221295), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.221319] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.221328] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.221336] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2292, ddl_arg=NULL) [2024-09-11 06:21:21.323553] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20125197) [2024-09-11 06:21:21.324361] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5441, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.324549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035681324115000, start_id=1726035681324115000, end_id=1726035681324115001) [2024-09-11 06:21:21.324943] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.324956] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.324964] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.325591] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325618] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325633] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.325645] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.325651] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.325655] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325667] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325678] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.325681] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.325684] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.325687] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.325692] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325703] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.325713] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.325716] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.325718] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.325720] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.325741] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681325723), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.325748] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.325753] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.325758] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2207, ddl_arg=NULL) [2024-09-11 06:21:21.427785] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=20020951) [2024-09-11 06:21:21.428571] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5442, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.428739] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035681428313000, start_id=1726035681428313000, end_id=1726035681428313001) [2024-09-11 06:21:21.429169] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.429184] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.429190] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.429853] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429880] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429897] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.429907] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.429913] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.429918] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429930] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429942] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.429945] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.429951] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.429955] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.429962] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429977] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.429989] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.429991] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.429994] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.429997] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.430017] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681430000), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.430025] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.430039] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.430043] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2260, ddl_arg=NULL) [2024-09-11 06:21:21.530290] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19918444) [2024-09-11 06:21:21.531073] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5443, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.531241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035681530300000, start_id=1726035681530300000, end_id=1726035681530300001) [2024-09-11 06:21:21.531715] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:21.531725] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.531732] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.531740] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.532397] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.532427] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.532443] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.532453] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.532462] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.532468] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.532486] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.532503] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:21.532508] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.532515] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.532544] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681532523), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.532555] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.532564] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.634745] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19813984) [2024-09-11 06:21:21.635547] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5444, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.635728] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035681635465000, start_id=1726035681635465000, end_id=1726035681635465001) [2024-09-11 06:21:21.636223] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.636235] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.636242] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.636922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.636951] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.636969] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.636982] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.636989] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.636993] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.637005] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.637017] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.637020] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.637023] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.637027] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.637030] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.637043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.637054] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.637057] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.637060] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.637063] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.637085] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681637067), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.637093] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.637100] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.637105] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2362, ddl_arg=NULL) [2024-09-11 06:21:21.739187] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19709544) [2024-09-11 06:21:21.739950] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5445, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.740107] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035681739700000, start_id=1726035681739700000, end_id=1726035681739700001) [2024-09-11 06:21:21.740543] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.740554] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.740563] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.741204] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741233] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741250] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.741259] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.741264] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.741269] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741282] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741293] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.741296] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.741299] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.741302] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.741307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741318] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.741329] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.741332] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.741334] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.741337] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.741357] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681741340), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.741364] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.741369] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.741374] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2190, ddl_arg=NULL) [2024-09-11 06:21:21.843600] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19605143) [2024-09-11 06:21:21.844388] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5446, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.844569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035681843945000, start_id=1726035681843945000, end_id=1726035681843945001) [2024-09-11 06:21:21.844989] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.845001] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.845007] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.845649] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845692] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.845703] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.845708] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.845715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845727] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845744] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.845749] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.845757] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.845764] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.845771] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845818] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=47] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.845831] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.845833] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.845835] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.845838] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.845862] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681845841), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.845870] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.845877] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.845884] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2287, ddl_arg=NULL) [2024-09-11 06:21:21.947872] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19500860) [2024-09-11 06:21:21.948674] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5447, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:21.948884] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035681948011000, start_id=1726035681948011000, end_id=1726035681948011001) [2024-09-11 06:21:21.949299] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:21.949316] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:21.949328] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.950137] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950172] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950193] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:21.950204] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:21.950210] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:21.950215] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950238] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:21.950241] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.950244] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.950254] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:21.950257] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950269] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:21.950279] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:21.950281] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:21.950285] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:21.950288] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:21.950311] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035681950293), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:21.950319] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:21.950325] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:21.950329] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2459, ddl_arg=NULL) [2024-09-11 06:21:22.062892] INFO [RS] add_server (ob_root_service.cpp:7915) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19385843) [2024-09-11 06:21:22.063680] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5448, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.063873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035682063568000, start_id=1726035682063568000, end_id=1726035682063568001) [2024-09-11 06:21:22.064270] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.064283] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.064291] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.064980] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065027] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.065040] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.065047] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.065054] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065077] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.065080] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.065085] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.065089] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.065094] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065105] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.065114] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.065116] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.065118] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.065120] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.065145] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682065124), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.065153] INFO [RS] add_server (ob_root_service.cpp:7928) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.065161] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.065166] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2276, ddl_arg=NULL) [2024-09-11 06:21:22.167380] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19281346) [2024-09-11 06:21:22.168219] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5449, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.168392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035682167780000, start_id=1726035682167780000, end_id=1726035682167780001) [2024-09-11 06:21:22.168844] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.168862] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.168873] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.169585] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169628] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.169642] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.169648] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.169655] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169665] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169674] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.169677] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.169683] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.169686] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.169691] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169702] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.169713] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.169716] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.169718] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.169720] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.169745] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682169724), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.169755] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.169766] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.169799] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] [DDL] execute ddl like stmt(ret=-4122, cost=2420, ddl_arg=NULL) [2024-09-11 06:21:22.271989] INFO [RS] add_server (ob_root_service.cpp:7915) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19176750) [2024-09-11 06:21:22.272734] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5450, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.272902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035682272036000, start_id=1726035682272036000, end_id=1726035682272036001) [2024-09-11 06:21:22.273330] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.273343] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.273349] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.274010] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274039] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274055] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.274065] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.274071] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.274077] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.274111] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.274115] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.274118] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.274121] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274131] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.274141] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.274144] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.274148] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.274152] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.274178] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682274159), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.274186] INFO [RS] add_server (ob_root_service.cpp:7928) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.274193] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.274199] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55188][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2211, ddl_arg=NULL) [2024-09-11 06:21:22.377133] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=19071598) [2024-09-11 06:21:22.378130] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5451, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.378360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035682377346000, start_id=1726035682377346000, end_id=1726035682377346001) [2024-09-11 06:21:22.378819] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.378837] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.378855] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.379715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379751] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379772] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.379794] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.379802] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.379811] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379830] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379852] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.379856] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.379861] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.379868] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.379873] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.379907] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.379911] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.379915] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.379922] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.379950] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682379929), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.379959] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.379970] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.379977] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2845, ddl_arg=NULL) [2024-09-11 06:21:22.502487] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18946247) [2024-09-11 06:21:22.503342] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5452, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.503537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035682502533000, start_id=1726035682502533000, end_id=1726035682502533001) [2024-09-11 06:21:22.503979] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:22.504636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504664] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504679] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.504693] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.504699] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.504704] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504717] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504729] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.504732] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.504736] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.504740] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.504746] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504758] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.504769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.504772] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.504785] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.504792] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.504817] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682504797), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.504824] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.504831] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2346, ddl_arg=NULL) [2024-09-11 06:21:22.607305] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18841434) [2024-09-11 06:21:22.608074] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5453, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.608248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035682607778000, start_id=1726035682607778000, end_id=1726035682607778001) [2024-09-11 06:21:22.608650] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:22.608660] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.608669] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.608676] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.609342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609371] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609387] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.609398] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.609404] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.609409] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609421] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609433] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.609436] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.609441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.609446] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.609449] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609461] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.609470] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.609474] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.609478] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.609484] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.609512] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682609488), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.609521] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.609527] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.609533] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:22.711812] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18736919) [2024-09-11 06:21:22.712651] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5454, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.712882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035682711963000, start_id=1726035682711963000, end_id=1726035682711963001) [2024-09-11 06:21:22.713291] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.713305] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.713313] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.714026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714070] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.714080] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.714086] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.714091] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714103] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714112] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.714115] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.714121] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.714125] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.714129] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714143] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.714159] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.714162] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.714165] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.714168] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.714194] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682714172), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.714208] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.714216] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.714221] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2410, ddl_arg=NULL) [2024-09-11 06:21:22.816367] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18632382) [2024-09-11 06:21:22.817122] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5455, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.817284] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035682817167000, start_id=1726035682817167000, end_id=1726035682817167001) [2024-09-11 06:21:22.817702] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.817716] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.817724] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.818366] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818395] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818411] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.818421] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.818427] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.818434] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818446] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818457] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.818460] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.818465] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.818468] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.818474] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.818496] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.818498] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.818501] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.818506] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.818528] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682818510), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.818536] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.818543] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.818548] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2183, ddl_arg=NULL) [2024-09-11 06:21:22.920645] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18528090) [2024-09-11 06:21:22.921433] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5456, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:22.921642] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:157) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] submit log success(service_type_=0, last_id=1726035699537832000, limited_id=1726035699537832000) [2024-09-11 06:21:22.921653] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1726035699537832000, limited_id=1726035699537832000) [2024-09-11 06:21:22.921657] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1726035682921406000, start_id=1726035682921406000, end_id=1726035682921406001) [2024-09-11 06:21:22.922160] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:22.922171] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:22.922180] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.922841] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922871] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922887] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:22.922899] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:22.922906] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:22.922911] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922931] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922942] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:22.922946] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.922949] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.922952] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:22.922956] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922969] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:22.922981] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:22.922984] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:22.922986] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:22.922989] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:22.923010] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035682922994), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:22.923019] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:22.923027] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:22.923031] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2388, ddl_arg=NULL) [2024-09-11 06:21:23.025228] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18423502) [2024-09-11 06:21:23.025995] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5457, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.026170] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035683025689000, start_id=1726035683025689000, end_id=1726035683025689001) [2024-09-11 06:21:23.026553] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.026566] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.026575] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.027261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027288] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681073157, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027304] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.027314] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.027321] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.027329] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027346] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027362] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.027366] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.027371] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.027379] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.027384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027396] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.027408] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:23.027411] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.027414] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.027417] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:23.027438] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683027421), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.027446] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.027454] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.027458] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2232, ddl_arg=NULL) [2024-09-11 06:21:23.129765] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18318961) [2024-09-11 06:21:23.130602] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5458, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.130807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035683130013000, start_id=1726035683130013000, end_id=1726035683130013001) [2024-09-11 06:21:23.131233] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.131251] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.131260] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.131945] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.131973] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.131988] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.132000] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.132010] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.132017] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.132028] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.132039] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.132042] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.132048] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.132051] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.132055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.132066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.132077] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:23.132080] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.132082] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.132088] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:23.132112] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683132093), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.132119] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.132124] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.132131] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2367, ddl_arg=NULL) [2024-09-11 06:21:23.234132] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=18214599) [2024-09-11 06:21:23.234901] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5459, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.235059] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035683234251000, start_id=1726035683234251000, end_id=1726035683234251001) [2024-09-11 06:21:23.235460] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.235472] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.235480] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.236112] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236144] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236160] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.236170] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.236176] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.236182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236194] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681300732, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236205] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.236209] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.236216] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.236221] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.236225] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035681284960, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.236245] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:23.236248] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:23.236250] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.236275] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683236253), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.236282] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.236289] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.338653] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:23.339538] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5460, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.339713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035683339552000, start_id=1726035683339552000, end_id=1726035683339552001) [2024-09-11 06:21:23.340133] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.340144] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.340151] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.340819] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.340846] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.340861] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.340870] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.340876] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.340884] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.340888] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.340914] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683340894), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.340919] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.340924] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.453847] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17994894) [2024-09-11 06:21:23.454666] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5461, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.454846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035683454155000, start_id=1726035683454155000, end_id=1726035683454155001) [2024-09-11 06:21:23.455279] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.455292] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.455301] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.455979] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.456006] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.456020] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.456032] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:23.456039] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.456045] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.456069] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683456048), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.456077] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.456084] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.558579] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17890153) [2024-09-11 06:21:23.559616] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5462, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.559907] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035683559388000, start_id=1726035683559388000, end_id=1726035683559388001) [2024-09-11 06:21:23.560403] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.560421] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.560433] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.561352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561392] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=39] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561417] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.561437] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.561446] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.561456] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561479] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561498] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.561505] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.561513] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.561518] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.561524] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561542] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.561559] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:23.561563] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.561570] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.561574] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:23.561604] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683561582), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.561616] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.561627] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.561633] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=3056, ddl_arg=NULL) [2024-09-11 06:21:23.673865] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17774876) [2024-09-11 06:21:23.674642] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5463, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.674828] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035683674120000, start_id=1726035683674120000, end_id=1726035683674120001) [2024-09-11 06:21:23.675280] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:23.675291] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.675300] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.675306] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.675958] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.675985] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.675999] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.676011] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.676016] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.676021] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.676032] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.676043] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.676045] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.676050] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.676072] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683676055), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.676080] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.676089] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.778353] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17670379) [2024-09-11 06:21:23.779236] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5464, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.779454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035683779175000, start_id=1726035683779175000, end_id=1726035683779175001) [2024-09-11 06:21:23.779939] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.779964] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.779975] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.780646] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.780672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.780688] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.780700] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.780707] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.780712] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.780723] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.780735] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.780738] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.780742] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.780745] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.780750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.780763] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.780811] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683780767), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.780821] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.780826] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.893282] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17555450) [2024-09-11 06:21:23.894096] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5465, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.894256] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035683893841000, start_id=1726035683893841000, end_id=1726035683893841001) [2024-09-11 06:21:23.894696] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.894706] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.894715] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.895394] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.895423] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.895446] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.895458] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.895464] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.895469] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.895481] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.895493] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.895496] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.895499] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.895505] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:23.895510] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.895513] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:23.895537] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683895517), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.895546] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.895555] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.997527] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17451202) [2024-09-11 06:21:23.998326] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5466, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:23.998501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035683998007000, start_id=1726035683998007000, end_id=1726035683998007001) [2024-09-11 06:21:23.998950] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:23.998965] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:23.998974] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.999642] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999692] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:23.999707] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:23.999713] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:23.999722] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999734] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999743] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:23.999746] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.999752] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.999756] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:23.999759] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999769] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:23.999803] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:23.999805] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:23.999808] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:23.999810] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:23.999847] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035683999818), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:23.999858] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:23.999864] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:23.999870] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2345, ddl_arg=NULL) [2024-09-11 06:21:24.112719] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17336011) [2024-09-11 06:21:24.113505] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5467, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.113696] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035684112732000, start_id=1726035684112732000, end_id=1726035684112732001) [2024-09-11 06:21:24.114121] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.114140] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.114146] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.114823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.114852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.114865] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.114874] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.114882] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.114890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.114915] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.114945] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.114948] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.114961] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.114969] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.114974] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.114990] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.115007] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.115014] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.115018] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.115021] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.115048] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684115028), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.115058] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.115069] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.115077] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2359, ddl_arg=NULL) [2024-09-11 06:21:24.217167] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17231561) [2024-09-11 06:21:24.217982] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5468, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.218156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035684218004000, start_id=1726035684218004000, end_id=1726035684218004001) [2024-09-11 06:21:24.218591] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.218603] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.218611] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.219270] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219311] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=41] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219327] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.219339] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.219345] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.219351] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219374] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.219377] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.219380] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.219387] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.219389] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219400] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.219408] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.219411] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.219413] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.219416] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.219437] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684219419), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.219444] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.219451] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.219458] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2293, ddl_arg=NULL) [2024-09-11 06:21:24.321662] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=17127062) [2024-09-11 06:21:24.322450] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5469, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.322624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035684322197000, start_id=1726035684322197000, end_id=1726035684322197001) [2024-09-11 06:21:24.323029] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.323042] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.323050] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.323681] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323708] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323723] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.323733] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.323739] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.323745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323769] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.323771] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.323782] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.323786] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.323790] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323807] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.323817] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.323820] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.323822] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.323827] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.323848] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684323831), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.323855] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.323862] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.323867] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2206, ddl_arg=NULL) [2024-09-11 06:21:24.426185] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:24.427239] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5470, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.427843] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.427859] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.427866] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.428484] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.428518] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.428534] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.428544] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.428555] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.428563] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.428585] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684428568), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.428593] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.428601] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.563315] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16885417) [2024-09-11 06:21:24.564138] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5471, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.564322] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035684563323000, start_id=1726035684563323000, end_id=1726035684563323001) [2024-09-11 06:21:24.564743] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.564755] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.564763] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.565433] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565463] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565479] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.565490] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.565496] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.565501] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565514] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565526] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.565529] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.565535] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.565538] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.565543] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565553] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.565563] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.565567] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.565572] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.565575] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.565595] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684565579), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.565601] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.565609] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.565614] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2299, ddl_arg=NULL) [2024-09-11 06:21:24.667942] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16780790) [2024-09-11 06:21:24.668687] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5472, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.668879] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035684668463000, start_id=1726035684668463000, end_id=1726035684668463001) [2024-09-11 06:21:24.669296] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.669309] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.669318] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:24.670001] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670025] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670039] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.670051] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.670056] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.670061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670075] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670085] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.670088] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.670103] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.670107] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.670111] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670123] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.670134] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.670157] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684670138), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.670166] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.772457] INFO [RS] add_server (ob_root_service.cpp:7915) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16676293) [2024-09-11 06:21:24.773485] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5473, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.773684] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035684773660000, start_id=1726035684773660000, end_id=1726035684773660001) [2024-09-11 06:21:24.774126] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:24.774140] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.774149] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.774164] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.775051] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775111] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.775125] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.775133] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.775142] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775168] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775186] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.775193] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.775200] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.775208] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.775213] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775230] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.775247] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.775253] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.775257] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.775264] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.775292] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684775271), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.775303] INFO [RS] add_server (ob_root_service.cpp:7928) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.775313] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.775321] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2867, ddl_arg=NULL) [2024-09-11 06:21:24.877189] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16571551) [2024-09-11 06:21:24.877992] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5474, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.878168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035684877911000, start_id=1726035684877911000, end_id=1726035684877911001) [2024-09-11 06:21:24.878601] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.878623] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.878631] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.879312] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879344] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879358] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.879371] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.879377] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.879383] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879399] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879412] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.879416] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.879419] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.879422] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.879426] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879437] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.879449] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.879451] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.879454] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.879456] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.879482] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684879460), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.879491] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.879498] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.879504] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:24.981558] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16467175) [2024-09-11 06:21:24.982353] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5475, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:24.982523] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035684982169000, start_id=1726035684982169000, end_id=1726035684982169001) [2024-09-11 06:21:24.982930] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:24.982943] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:24.982951] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.983590] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983620] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683073477, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983636] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:24.983648] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:24.983654] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:24.983661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983672] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983684] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:24.983686] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.983692] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.983695] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:24.983705] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983717] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:24.983727] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:24.983730] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:24.983733] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:24.983735] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:24.983760] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035684983741), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:24.983769] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:24.983784] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:24.983787] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2231, ddl_arg=NULL) [2024-09-11 06:21:25.086205] INFO [RS] add_server (ob_root_service.cpp:7915) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16362523) [2024-09-11 06:21:25.087006] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5476, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.087252] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035685086462000, start_id=1726035685086462000, end_id=1726035685086462001) [2024-09-11 06:21:25.087664] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.087677] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.087686] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.088363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088397] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088413] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.088425] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.088431] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.088438] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088449] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088461] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.088463] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.088467] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.088472] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.088475] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.088496] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.088498] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.088501] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.088503] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.088530] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685088507), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.088540] INFO [RS] add_server (ob_root_service.cpp:7928) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.088545] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.088549] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55179][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2346, ddl_arg=NULL) [2024-09-11 06:21:25.190640] INFO [RS] add_server (ob_root_service.cpp:7915) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16258088) [2024-09-11 06:21:25.191405] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5477, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.191573] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1726035685190540000, start_id=1726035685190540000, end_id=1726035685190540001) [2024-09-11 06:21:25.191986] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.191999] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.192012] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.192658] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192702] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.192716] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.192723] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.192731] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192745] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.192759] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.192761] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.192767] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.192770] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192788] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683285308, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.192797] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.192802] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.192804] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.192807] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.192831] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685192812), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.192838] INFO [RS] add_server (ob_root_service.cpp:7928) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.192845] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.192852] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55226][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2214, ddl_arg=NULL) [2024-09-11 06:21:25.295335] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16153398) [2024-09-11 06:21:25.296194] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5478, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.296369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035685295766000, start_id=1726035685295766000, end_id=1726035685295766001) [2024-09-11 06:21:25.296781] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.296795] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.296802] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.297476] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297507] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297526] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.297537] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.297543] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.297548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297559] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035683301124, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297572] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.297574] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.297579] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.297585] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.297589] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297600] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.297611] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.297614] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.297617] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.297619] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.297642] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685297622), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.297650] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.297656] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.297661] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2327, ddl_arg=NULL) [2024-09-11 06:21:25.399850] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=16048895) [2024-09-11 06:21:25.400666] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5479, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.400861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035685399853000, start_id=1726035685399853000, end_id=1726035685399853001) [2024-09-11 06:21:25.401264] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.401278] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.401287] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.401977] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402005] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402018] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.402029] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.402035] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.402042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402053] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402064] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.402067] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.402073] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.402076] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.402080] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402096] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.402108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.402110] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.402113] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.402117] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.402147] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685402121), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.402158] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.402167] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.402175] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2327, ddl_arg=NULL) [2024-09-11 06:21:25.504442] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15944291) [2024-09-11 06:21:25.505253] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5480, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.505438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035685505016000, start_id=1726035685505016000, end_id=1726035685505016001) [2024-09-11 06:21:25.505941] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.505952] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.505961] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.506686] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.506723] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.506740] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.506752] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:25.506764] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.506783] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.506808] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685506791), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.506817] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.506824] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.608857] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15839878) [2024-09-11 06:21:25.609625] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5481, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.609783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035685609197000, start_id=1726035685609197000, end_id=1726035685609197001) [2024-09-11 06:21:25.610195] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.610207] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.610213] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.610844] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610871] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610887] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.610897] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.610902] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.610909] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610921] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610933] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.610935] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.610939] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.610942] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.610947] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.610969] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.610972] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.610974] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.610977] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.610998] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685610982), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.611006] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.611012] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.611018] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2163, ddl_arg=NULL) [2024-09-11 06:21:25.713622] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15735108) [2024-09-11 06:21:25.714409] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5482, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.714627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:25.715061] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.715075] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.715725] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.715754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.715788] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.715804] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.715814] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:25.715868] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685715821), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.715883] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.818117] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15630616) [2024-09-11 06:21:25.818882] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5483, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.819050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035685818362000, start_id=1726035685818362000, end_id=1726035685818362001) [2024-09-11 06:21:25.819532] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:25.820189] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.820216] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.820234] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.820249] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.820254] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.820262] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.820274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.820286] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.820289] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.820315] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685820296), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.820324] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.933295] INFO [RS] add_server (ob_root_service.cpp:7915) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15515425) [2024-09-11 06:21:25.934131] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5484, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:25.934324] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035685934167000, start_id=1726035685934167000, end_id=1726035685934167001) [2024-09-11 06:21:25.934770] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:25.934809] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:25.934818] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.935473] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935501] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935517] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:25.935528] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:25.935534] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:25.935539] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935562] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:25.935565] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.935571] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:25.935576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935587] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:25.935598] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:25.935601] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:25.935604] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:25.935629] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035685935608), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:25.935642] INFO [RS] add_server (ob_root_service.cpp:7928) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:25.935647] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:25.935652] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55212][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2359, ddl_arg=NULL) [2024-09-11 06:21:26.037905] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15410836) [2024-09-11 06:21:26.038617] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5485, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.038819] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035686038285000, start_id=1726035686038285000, end_id=1726035686038285001) [2024-09-11 06:21:26.039261] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.039275] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.039282] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.039938] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.039970] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.039985] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.039998] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.040004] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.040011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.040022] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.040033] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.040036] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.040039] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.040046] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.040053] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.040071] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.040083] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.040086] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.040088] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.040108] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686040092), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.040116] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.040123] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.040127] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2224, ddl_arg=NULL) [2024-09-11 06:21:26.152962] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15295770) [2024-09-11 06:21:26.153758] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5486, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.153965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1726035686152950000, start_id=1726035686152950000, end_id=1726035686152950001) [2024-09-11 06:21:26.154365] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.154377] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.154385] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.155066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155095] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.155123] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.155130] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.155137] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155147] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155163] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.155166] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.155173] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.155181] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.155189] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155204] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.155213] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.155216] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.155218] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.155244] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686155224), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.155252] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.155260] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.155266] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2307, ddl_arg=NULL) [2024-09-11 06:21:26.258752] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.258767] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.258785] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.259490] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686259466), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.259503] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.259510] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.362524] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=15086211) [2024-09-11 06:21:26.363362] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5488, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.363552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1726035686363194000, start_id=1726035686363194000, end_id=1726035686363194001) [2024-09-11 06:21:26.363979] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.363991] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.364000] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.364660] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364689] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364707] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.364722] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.364728] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.364735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364746] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364756] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.364759] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.364763] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.364780] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.364783] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364793] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.364802] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.364804] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.364807] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.364827] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686364810), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.364834] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.364841] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.364846] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2324, ddl_arg=NULL) [2024-09-11 06:21:26.466913] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14981847) [2024-09-11 06:21:26.467692] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5489, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.467860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035686467511000, start_id=1726035686467511000, end_id=1726035686467511001) [2024-09-11 06:21:26.468265] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.468278] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.468289] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.468997] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.469027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.469046] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.469057] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.469064] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.469068] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.469079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.469090] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.469093] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.469096] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.469120] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686469103), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.469127] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.469132] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.571452] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14877279) [2024-09-11 06:21:26.572300] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5490, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.572480] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035686571642000, start_id=1726035686571642000, end_id=1726035686571642001) [2024-09-11 06:21:26.572914] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.572929] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.572936] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.573610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573644] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573659] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.573671] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.573678] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.573687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573699] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573709] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.573712] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.573718] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.573723] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.573726] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573738] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.573750] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.573753] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.573756] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.573794] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686573760), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.573805] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.573811] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.573816] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2366, ddl_arg=NULL) [2024-09-11 06:21:26.677168] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14771573) [2024-09-11 06:21:26.677940] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5491, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.678115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035686677959000, start_id=1726035686677959000, end_id=1726035686677959001) [2024-09-11 06:21:26.678572] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.678586] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.678593] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.679287] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.679312] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:26.679336] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.679344] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.679364] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686679347), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.679372] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.679380] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.782089] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14666645) [2024-09-11 06:21:26.782943] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5492, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.783142] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035686783116000, start_id=1726035686783116000, end_id=1726035686783116001) [2024-09-11 06:21:26.783634] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.783647] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.783655] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.784317] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784343] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784358] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.784369] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.784375] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.784382] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784404] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784415] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.784418] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.784421] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.784424] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.784426] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.784446] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.784449] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.784451] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.784456] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.784474] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686784459), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.784480] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.784487] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.784491] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2404, ddl_arg=NULL) [2024-09-11 06:21:26.886766] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14561965) [2024-09-11 06:21:26.887525] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5493, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.887694] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035686887139000, start_id=1726035686887139000, end_id=1726035686887139001) [2024-09-11 06:21:26.888091] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:26.888101] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.888108] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.888116] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.888731] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888806] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.888820] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.888826] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.888831] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888842] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888854] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.888857] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.888860] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.888864] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.888867] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888879] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.888888] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.888890] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.888892] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.888894] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.888915] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686888898), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.888924] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.888929] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.888935] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2170, ddl_arg=NULL) [2024-09-11 06:21:26.991688] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14457052) [2024-09-11 06:21:26.992495] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5494, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:26.992668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035686992328000, start_id=1726035686992328000, end_id=1726035686992328001) [2024-09-11 06:21:26.993112] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:26.993126] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:26.993135] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.993799] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993828] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685073819, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993844] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:26.993855] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:26.993861] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:26.993868] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993879] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993888] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:26.993891] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.993897] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.993900] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:26.993905] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993917] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:26.993926] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:26.993929] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:26.993931] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:26.993934] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:26.993963] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035686993939), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:26.993971] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:26.993978] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:26.993984] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2298, ddl_arg=NULL) [2024-09-11 06:21:27.096435] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14352288) [2024-09-11 06:21:27.097232] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5495, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.097387] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035687096500000, start_id=1726035687096500000, end_id=1726035687096500001) [2024-09-11 06:21:27.097858] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.097874] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.097884] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.098603] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098632] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098646] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.098657] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.098662] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.098667] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098680] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098692] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:27.098695] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.098700] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.098706] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:27.098710] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098721] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685285662, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.098731] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:27.098735] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.098739] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.098743] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:27.098770] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687098750), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.098801] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.098807] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.098811] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2378, ddl_arg=NULL) [2024-09-11 06:21:27.201401] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14247343) [2024-09-11 06:21:27.202221] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5496, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.202388] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035687201779000, start_id=1726035687201779000, end_id=1726035687201779001) [2024-09-11 06:21:27.202820] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.202834] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.202844] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.203504] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.203533] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.203549] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.203560] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.203566] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.203572] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.203583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035685301525, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.203594] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:27.203597] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.203603] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.203614] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:27.203619] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.203641] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687203622), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.203649] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.203656] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.306118] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=14142615) [2024-09-11 06:21:27.306902] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5497, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.307099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035687306779000, start_id=1726035687306779000, end_id=1726035687306779001) [2024-09-11 06:21:27.307506] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.307520] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.307527] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.308220] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308263] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.308273] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.308279] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.308284] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308297] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308308] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:27.308311] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.308317] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.308321] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:27.308325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308336] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.308347] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:27.308352] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.308355] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.308358] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:27.308382] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687308361), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.308390] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.308396] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.308402] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2286, ddl_arg=NULL) [2024-09-11 06:21:27.411488] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5498, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.411676] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035687410894000, start_id=1726035687410894000, end_id=1726035687410894001) [2024-09-11 06:21:27.412140] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.412154] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.412162] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.412822] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.412832] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.412853] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687412836), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.412861] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.412869] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.515891] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13932846) [2024-09-11 06:21:27.516725] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5499, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.516965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035687516269000, start_id=1726035687516269000, end_id=1726035687516269001) [2024-09-11 06:21:27.517453] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.517467] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.517475] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.518249] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518279] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518294] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.518306] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.518314] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.518321] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518333] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518344] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:27.518346] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.518352] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.518355] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:27.518360] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518372] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.518384] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:27.518386] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:27.518388] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.518390] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:27.518414] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687518395), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.518422] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.518429] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.621077] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13827647) [2024-09-11 06:21:27.621963] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5500, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.622196] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035687621462000, start_id=1726035687621462000, end_id=1726035687621462001) [2024-09-11 06:21:27.623031] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.623050] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.623060] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.623753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.623792] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.623814] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.623828] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.623837] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.623844] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.623863] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.623880] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.623889] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.623921] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687623897), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.623932] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.623940] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.725970] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13722771) [2024-09-11 06:21:27.726822] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5501, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.727021] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035687726750000, start_id=1726035687726750000, end_id=1726035687726750001) [2024-09-11 06:21:27.727437] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.727455] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.727464] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.728216] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.728251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.728268] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.728280] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.728284] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.728291] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.728307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:27.728327] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.728333] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.728356] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687728337), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.728364] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.728371] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.831315] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13617412) [2024-09-11 06:21:27.832082] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5502, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:27.832260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035687831782000, start_id=1726035687831782000, end_id=1726035687831782001) [2024-09-11 06:21:27.832695] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.832709] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.832717] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.833380] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.833410] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.833425] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.833434] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:27.833440] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:27.833448] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.833455] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.833475] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687833458), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.833482] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.833490] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:27.833494] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2180, ddl_arg=NULL) [2024-09-11 06:21:27.936399] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13512326) [2024-09-11 06:21:27.937758] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:27.937770] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:27.937798] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:27.937806] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.938478] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.938509] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:27.938524] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:27.938537] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.938544] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:27.938567] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035687938547), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:27.938576] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:27.938584] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.041425] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13407310) [2024-09-11 06:21:28.042219] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5504, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.042394] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035688042141000, start_id=1726035688042141000, end_id=1726035688042141001) [2024-09-11 06:21:28.042825] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.042841] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.042850] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.043537] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.043570] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.043586] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.043598] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.043604] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.043611] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.043626] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:28.043631] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:28.043656] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688043636), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.043665] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.043672] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.146571] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13302157) [2024-09-11 06:21:28.147394] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5505, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.147571] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.147995] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.148010] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.148025] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.148721] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688148699), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.148735] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.148743] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.251815] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=13196912) [2024-09-11 06:21:28.252547] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.253162] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.253173] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.253183] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.253846] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.253877] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.253891] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.253938] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688253919), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.253947] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.253954] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.357595] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.358372] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5507, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.358536] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035688357640000, start_id=1726035688357640000, end_id=1726035688357640001) [2024-09-11 06:21:28.358946] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.358962] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.358970] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.359627] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.359656] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.359671] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.359682] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.359688] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.359718] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688359698), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.359727] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.359733] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.463652] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.464436] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5508, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.464599] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035688463986000, start_id=1726035688463986000, end_id=1726035688463986001) [2024-09-11 06:21:28.465039] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.465052] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.465061] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.465724] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.465752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.465826] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688465805), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.465833] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.465840] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.569622] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12879109) [2024-09-11 06:21:28.570516] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5509, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.570732] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035688570257000, start_id=1726035688570257000, end_id=1726035688570257001) [2024-09-11 06:21:28.571281] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.571296] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.571304] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.571965] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.571996] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.572012] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.572025] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.572032] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.572037] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.572049] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.572083] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688572064), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.572091] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.572097] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.674384] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12774340) [2024-09-11 06:21:28.675204] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5510, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.675372] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035688674440000, start_id=1726035688674440000, end_id=1726035688674440001) [2024-09-11 06:21:28.675986] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.675999] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.676008] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.676663] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676692] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676706] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.676718] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.676724] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.676729] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676740] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676751] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:28.676754] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.676760] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:28.676763] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:28.676766] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676786] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.676799] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:28.676805] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.676808] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:28.676810] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:28.676835] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688676815), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.676843] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.676851] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.676857] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2475, ddl_arg=NULL) [2024-09-11 06:21:28.779331] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12669399) [2024-09-11 06:21:28.780078] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5511, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.780247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035688779644000, start_id=1726035688779644000, end_id=1726035688779644001) [2024-09-11 06:21:28.780671] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.780686] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.780695] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.781380] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781408] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781424] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.781436] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.781442] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.781447] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781458] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781469] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:28.781471] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.781475] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:28.781479] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:28.781482] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781493] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.781504] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:28.781507] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.781509] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:28.781531] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688781512), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.781538] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.781543] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.884204] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12564524) [2024-09-11 06:21:28.884958] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5512, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.885143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035688884759000, start_id=1726035688884759000, end_id=1726035688884759001) [2024-09-11 06:21:28.885663] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.885676] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.885685] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.886321] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.886349] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.886396] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688886375), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.886408] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.886420] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.988989] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12459750) [2024-09-11 06:21:28.989715] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5513, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:28.989911] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035688988949000, start_id=1726035688988949000, end_id=1726035688988949001) [2024-09-11 06:21:28.990327] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:28.990339] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:28.990349] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:28.990359] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.991014] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991042] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687074159, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991056] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:28.991066] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] added async task to update rslist(force_update=false) [2024-09-11 06:21:28.991072] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:28.991080] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991110] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:28.991112] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.991116] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:28.991121] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991131] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:28.991145] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:28.991148] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:28.991150] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:28.991169] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035688991153), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:28.991175] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:28.991182] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:28.991190] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2202, ddl_arg=NULL) [2024-09-11 06:21:29.093605] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12355126) [2024-09-11 06:21:29.094439] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5514, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.094602] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035689094217000, start_id=1726035689094217000, end_id=1726035689094217001) [2024-09-11 06:21:29.095025] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.095040] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:29.095679] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095708] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095729] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.095741] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.095747] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.095754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095770] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095791] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.095794] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.095798] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.095803] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095815] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.095826] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.095828] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.095830] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.095856] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689095837), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.095864] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.095873] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2271, ddl_arg=NULL) [2024-09-11 06:21:29.198172] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12250556) [2024-09-11 06:21:29.198985] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5515, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.199175] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035689198525000, start_id=1726035689198525000, end_id=1726035689198525001) [2024-09-11 06:21:29.200236] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200264] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200286] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.200297] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.200303] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.200310] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200323] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687301961, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200334] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.200336] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.200341] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.200346] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200357] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035687286025, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.200368] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.200370] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.200373] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.200403] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689200379), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.200411] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.200420] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] [DDL] execute ddl like stmt(ret=-4122, cost=2251, ddl_arg=NULL) [2024-09-11 06:21:29.302747] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12145993) [2024-09-11 06:21:29.303548] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5516, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.303721] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035689303692000, start_id=1726035689303692000, end_id=1726035689303692001) [2024-09-11 06:21:29.304166] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.304177] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.304184] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.304869] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.304898] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.304913] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.304924] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.304930] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.304937] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.304949] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.304958] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.304961] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.304967] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:29.304971] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.304976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.304992] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.305005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.305008] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.305010] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.305033] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689305014), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.305040] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.305048] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.305052] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2306, ddl_arg=NULL) [2024-09-11 06:21:29.407077] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=12041650) [2024-09-11 06:21:29.407887] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5517, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.408063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035689407780000, start_id=1726035689407780000, end_id=1726035689407780001) [2024-09-11 06:21:29.408597] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.408611] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.408620] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.409293] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409325] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409347] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.409363] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.409372] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.409379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409394] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409405] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.409408] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.409414] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.409420] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.409445] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.409447] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.409450] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.409473] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689409454), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.409481] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.409487] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.409496] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [DDL] execute ddl like stmt(ret=-4122, cost=2421, ddl_arg=NULL) [2024-09-11 06:21:29.511667] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11937066) [2024-09-11 06:21:29.512430] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5518, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.512589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035689511903000, start_id=1726035689511903000, end_id=1726035689511903001) [2024-09-11 06:21:29.513245] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.513256] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.513263] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.513934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.513965] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.513979] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.513989] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.513994] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.513998] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.514008] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.514020] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.514023] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.514026] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:29.514031] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.514034] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.514045] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.514054] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.514057] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.514062] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.514086] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689514065), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.514095] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.514105] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.616261] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11832462) [2024-09-11 06:21:29.617107] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5519, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.617289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035689617005000, start_id=1726035689617005000, end_id=1726035689617005001) [2024-09-11 06:21:29.617698] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.617713] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.617722] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.618403] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618432] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618446] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.618458] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.618464] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.618469] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618480] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618491] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.618494] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.618500] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:29.618505] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618522] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.618533] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:29.618537] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.618540] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:29.618567] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689618545), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.618579] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.618589] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.618595] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2336, ddl_arg=NULL) [2024-09-11 06:21:29.720671] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11728061) [2024-09-11 06:21:29.721473] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:29.722236] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.722250] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.722259] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.722929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.722959] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.722982] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.722990] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.723018] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689722997), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.723027] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.723035] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.825458] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11623266) [2024-09-11 06:21:29.826227] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5521, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.826375] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035689825595000, start_id=1726035689825595000, end_id=1726035689825595001) [2024-09-11 06:21:29.826758] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.826770] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.826787] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.827421] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.827450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.827471] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.827484] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.827491] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.827497] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.827508] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.827520] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:29.827523] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:29.827527] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.827532] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:29.827537] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.827558] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689827541), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.827566] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.827573] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.929838] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11518905) [2024-09-11 06:21:29.930568] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5522, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:29.930743] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035689929780000, start_id=1726035689929780000, end_id=1726035689929780001) [2024-09-11 06:21:29.931183] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:29.931195] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:29.931203] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.931878] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.931906] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.931919] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:29.931929] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:29.931933] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:29.931937] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:29.931949] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.931958] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:29.931979] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035689931963), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:29.931986] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:29.931992] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:29.931996] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2160, ddl_arg=NULL) [2024-09-11 06:21:30.034411] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11414315) [2024-09-11 06:21:30.035424] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5523, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.035597] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035690034960000, start_id=1726035690034960000, end_id=1726035690034960001) [2024-09-11 06:21:30.036063] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:30.036074] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.036082] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.036087] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.036725] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036753] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036772] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.036797] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.036803] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.036810] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036823] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036834] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.036837] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.036840] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.036843] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.036845] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036857] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.036868] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.036870] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.036872] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.036875] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.036894] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690036878), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.036902] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.036909] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.036912] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2503, ddl_arg=NULL) [2024-09-11 06:21:30.139217] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11309516) [2024-09-11 06:21:30.140007] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5524, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.140194] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035690139185000, start_id=1726035690139185000, end_id=1726035690139185001) [2024-09-11 06:21:30.140635] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.140646] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.140654] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.141303] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141336] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141351] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.141360] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.141366] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.141373] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141395] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.141397] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.141401] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.141406] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.141409] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141420] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.141431] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.141434] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.141441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.141444] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.141469] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690141451), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.141476] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.141483] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.141487] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2271, ddl_arg=NULL) [2024-09-11 06:21:30.253760] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11194973) [2024-09-11 06:21:30.254517] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5525, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.254684] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035690254681000, start_id=1726035690254681000, end_id=1726035690254681001) [2024-09-11 06:21:30.255124] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.255136] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:30.255752] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255812] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=59] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255826] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.255837] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.255843] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.255849] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255860] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255873] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.255877] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.255880] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.255886] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.255890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255902] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.255918] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.255921] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.255924] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.255927] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.255948] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690255931), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.255957] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.255964] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2206, ddl_arg=NULL) [2024-09-11 06:21:30.358349] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=11090382) [2024-09-11 06:21:30.359085] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5526, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.359244] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035690358888000, start_id=1726035690358888000, end_id=1726035690358888001) [2024-09-11 06:21:30.359667] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.359680] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.359689] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.360337] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360365] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360381] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.360394] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.360401] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.360407] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360424] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360436] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.360439] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.360442] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.360447] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.360450] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360462] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.360474] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.360478] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.360482] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.360487] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.360512] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690360494), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.360520] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.360529] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.360535] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:30.463031] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10985707) [2024-09-11 06:21:30.463910] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5527, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.464129] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035690464116000, start_id=1726035690464116000, end_id=1726035690464116001) [2024-09-11 06:21:30.464708] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.464726] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.464737] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.465527] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.465557] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.465577] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.465587] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.465593] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.465598] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.465610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.465621] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.465624] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.465627] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.465632] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.465654] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690465638), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.465660] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.465667] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.567634] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10881093) [2024-09-11 06:21:30.568383] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5528, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.568529] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035690568117000, start_id=1726035690568117000, end_id=1726035690568117001) [2024-09-11 06:21:30.568991] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.569006] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.569015] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.569704] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569734] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569750] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.569762] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.569768] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.569782] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569810] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.569813] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.569816] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.569820] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.569824] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569835] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.569850] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.569853] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.569856] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.569859] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.569885] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690569863), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.569893] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.569903] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.569908] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2275, ddl_arg=NULL) [2024-09-11 06:21:30.673647] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10775075) [2024-09-11 06:21:30.674485] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5529, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.674671] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035690674430000, start_id=1726035690674430000, end_id=1726035690674430001) [2024-09-11 06:21:30.675042] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.675055] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.675064] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.675737] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675772] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675794] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.675807] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.675813] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.675820] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675832] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675847] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:30.675850] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.675856] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.675860] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:30.675865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675876] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.675887] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:30.675890] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:30.675892] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.675894] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:30.675918] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690675897), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.675926] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.675933] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.675938] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2293, ddl_arg=NULL) [2024-09-11 06:21:30.800421] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5530, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.800596] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035690799728000, start_id=1726035690799728000, end_id=1726035690799728001) [2024-09-11 06:21:30.801018] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.801031] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.801038] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.801702] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.801735] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.801758] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.801766] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.801804] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690801769), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.801815] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.801825] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:30.904837] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10543917) [2024-09-11 06:21:30.905596] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5531, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:30.905771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035690904894000, start_id=1726035690904894000, end_id=1726035690904894001) [2024-09-11 06:21:30.906233] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=55][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:30.906246] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:30.906255] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.906949] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.906985] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:30.907002] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:30.907015] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:30.907022] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:30.907027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:30.907042] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.907047] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:30.907071] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035690907053), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:30.907079] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:30.907084] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.009493] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10439228) [2024-09-11 06:21:31.010261] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5532, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.010459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035691010074000, start_id=1726035691010074000, end_id=1726035691010074001) [2024-09-11 06:21:31.010880] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.010892] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.010898] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.011523] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011550] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689074511, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011566] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.011575] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.011584] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.011590] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011605] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011621] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.011625] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.011632] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.011637] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.011642] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011658] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.011674] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.011678] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.011682] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.011686] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.011715] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691011694), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.011726] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.011736] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.011742] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2250, ddl_arg=NULL) [2024-09-11 06:21:31.113961] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10334770) [2024-09-11 06:21:31.114789] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5533, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.114978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035691114286000, start_id=1726035691114286000, end_id=1726035691114286001) [2024-09-11 06:21:31.115414] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:31.115425] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.115434] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.115441] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.116143] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116173] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116190] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.116201] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.116207] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.116219] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116231] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116242] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.116244] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.116247] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.116253] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.116255] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116267] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.116277] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.116279] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.116281] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.116283] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.116307] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691116287), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.116315] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.116324] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.116328] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2374, ddl_arg=NULL) [2024-09-11 06:21:31.218268] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10230469) [2024-09-11 06:21:31.219048] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5534, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.219207] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035691218412000, start_id=1726035691218412000, end_id=1726035691218412001) [2024-09-11 06:21:31.219616] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.219630] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.219637] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.220304] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220334] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220347] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.220359] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.220365] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.220372] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220383] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689302369, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220395] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.220398] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.220401] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.220404] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.220407] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220418] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035689286498, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.220428] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.220431] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.220433] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.220435] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.220454] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691220438), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.220461] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.220471] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.220476] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2209, ddl_arg=NULL) [2024-09-11 06:21:31.333063] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10115672) [2024-09-11 06:21:31.333846] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5535, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.334014] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035691333235000, start_id=1726035691333235000, end_id=1726035691333235001) [2024-09-11 06:21:31.334457] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.334469] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.334477] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.335176] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335204] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335218] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.335230] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.335234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.335238] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335248] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335260] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.335262] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.335266] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.335271] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.335274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.335294] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.335297] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.335299] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.335301] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.335320] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691335304), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.335327] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.335335] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.335339] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2278, ddl_arg=NULL) [2024-09-11 06:21:31.437993] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=10010733) [2024-09-11 06:21:31.438802] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5536, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.438997] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035691438359000, start_id=1726035691438359000, end_id=1726035691438359001) [2024-09-11 06:21:31.439434] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.439448] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.439457] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.440125] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440157] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440170] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.440182] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.440188] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.440195] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440208] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440219] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.440222] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.440225] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.440231] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.440234] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440244] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.440255] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.440258] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.440260] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.440263] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.440286] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691440266), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.440295] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.440302] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.440310] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2319, ddl_arg=NULL) [2024-09-11 06:21:31.553394] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9895338) [2024-09-11 06:21:31.554164] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5537, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.554333] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035691554069000, start_id=1726035691554069000, end_id=1726035691554069001) [2024-09-11 06:21:31.554718] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.554737] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.554749] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.555583] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555627] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.555638] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.555644] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.555651] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555671] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.555676] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.555679] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.555684] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.555687] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555697] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.555708] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.555712] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.555715] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.555719] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.555753] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691555726), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.555764] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.555799] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.555808] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2417, ddl_arg=NULL) [2024-09-11 06:21:31.657966] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9790767) [2024-09-11 06:21:31.658755] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5538, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.658953] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035691658123000, start_id=1726035691658123000, end_id=1726035691658123001) [2024-09-11 06:21:31.659369] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.659381] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.659389] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.660055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660085] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660101] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.660111] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.660116] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.660120] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660133] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660144] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.660147] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.660152] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.660156] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.660159] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660170] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.660192] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:31.660195] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.660225] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691660200), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.660234] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.660242] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.773062] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9675662) [2024-09-11 06:21:31.773870] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5539, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.774051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035691773844000, start_id=1726035691773844000, end_id=1726035691773844001) [2024-09-11 06:21:31.774490] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.774503] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.774511] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.775228] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775259] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775274] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.775284] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.775290] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.775297] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775308] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775320] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.775322] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.775325] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.775331] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.775334] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775346] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.775359] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.775361] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.775364] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.775366] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.775389] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691775369), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.775398] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.775405] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.775412] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2352, ddl_arg=NULL) [2024-09-11 06:21:31.877639] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9571094) [2024-09-11 06:21:31.878486] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5540, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.878674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035691878146000, start_id=1726035691878146000, end_id=1726035691878146001) [2024-09-11 06:21:31.879151] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.879164] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.879172] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.879796] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879825] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879838] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.879850] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.879857] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.879862] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879874] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879886] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.879889] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.879892] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.879896] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.879902] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879912] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.879924] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.879927] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.879930] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.879935] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.879956] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691879939), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.879962] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.879971] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.879975] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2338, ddl_arg=NULL) [2024-09-11 06:21:31.982146] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9466595) [2024-09-11 06:21:31.982907] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5541, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:31.983053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1726035691982242000, start_id=1726035691982242000, end_id=1726035691982242001) [2024-09-11 06:21:31.983502] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:31.983515] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:31.983523] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.984189] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984217] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984231] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:31.984244] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:31.984250] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:31.984255] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984286] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:31.984289] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.984293] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.984297] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:31.984301] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984312] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:31.984323] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:31.984326] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:31.984328] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:31.984331] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:31.984352] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035691984335), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:31.984360] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:31.984366] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:31.984369] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2225, ddl_arg=NULL) [2024-09-11 06:21:32.086882] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9361851) [2024-09-11 06:21:32.087681] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5542, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.087872] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035692087565000, start_id=1726035692087565000, end_id=1726035692087565001) [2024-09-11 06:21:32.088287] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.088299] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.088306] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.088982] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089017] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089033] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.089044] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.089050] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.089054] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089078] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.089081] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.089087] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.089091] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.089095] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089104] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.089116] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.089118] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.089120] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.089122] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.089143] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692089125), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.089151] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.089158] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.089162] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2283, ddl_arg=NULL) [2024-09-11 06:21:32.191282] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9257448) [2024-09-11 06:21:32.192214] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5543, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.192376] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035692191839000, start_id=1726035692191839000, end_id=1726035692191839001) [2024-09-11 06:21:32.192796] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:32.192806] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.192813] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.192821] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.193570] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193608] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193633] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.193647] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.193656] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.193666] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193688] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193708] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.193714] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.193722] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.193728] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.193733] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193754] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.193805] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=51] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.193813] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.193818] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.193826] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.193851] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692193831), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.193862] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.193872] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.193881] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=2601, ddl_arg=NULL) [2024-09-11 06:21:32.295856] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9152863) [2024-09-11 06:21:32.296718] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5544, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.296914] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035692295892000, start_id=1726035692295892000, end_id=1726035692295892001) [2024-09-11 06:21:32.297324] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.297340] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.297352] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.298104] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298134] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.298164] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.298171] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.298176] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298189] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298201] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.298205] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.298211] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.298215] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.298218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298229] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.298240] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.298243] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.298246] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.298249] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.298274] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692298254), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.298284] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.298294] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.298301] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2448, ddl_arg=NULL) [2024-09-11 06:21:32.400560] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=9048160) [2024-09-11 06:21:32.401395] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5545, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.401572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035692401051000, start_id=1726035692401051000, end_id=1726035692401051001) [2024-09-11 06:21:32.402004] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.402032] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.402041] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.402679] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.402708] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.402723] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.402733] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.402739] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.402744] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.402755] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.402766] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.402768] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.402772] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.402787] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.402794] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.402819] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692402799), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.402824] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.402829] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.505068] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8943656) [2024-09-11 06:21:32.505875] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5546, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.506068] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035692505434000, start_id=1726035692505434000, end_id=1726035692505434001) [2024-09-11 06:21:32.507213] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507243] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507259] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.507269] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.507277] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.507283] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507299] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507315] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.507319] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.507327] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.507332] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.507341] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507358] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.507375] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.507379] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.507383] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.507389] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.507420] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692507397), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.507429] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.507441] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2375, ddl_arg=NULL) [2024-09-11 06:21:32.609832] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8838906) [2024-09-11 06:21:32.610607] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5547, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.610764] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035692610680000, start_id=1726035692610680000, end_id=1726035692610680001) [2024-09-11 06:21:32.611183] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.611196] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.611202] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.611875] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611903] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611916] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.611926] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.611932] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.611937] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611950] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611961] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.611964] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.611968] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.611971] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.611975] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611984] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.611995] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.611999] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.612001] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.612003] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.612028] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692612007), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.612036] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.612042] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.612045] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:32.715100] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5548, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.715703] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.715718] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.715727] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.716443] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.716481] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.716487] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.716509] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692716491), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.716518] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.716524] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.819582] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8629149) [2024-09-11 06:21:32.820421] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5549, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.820602] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035692820115000, start_id=1726035692820115000, end_id=1726035692820115001) [2024-09-11 06:21:32.821041] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.821054] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.821063] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.821730] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821760] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821796] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.821811] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.821817] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.821821] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821834] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821843] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.821846] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.821851] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.821855] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.821860] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821872] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.821883] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.821886] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.821888] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.821890] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.821915] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692821894), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.821923] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.821931] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.821937] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2358, ddl_arg=NULL) [2024-09-11 06:21:32.924110] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8524624) [2024-09-11 06:21:32.924923] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5550, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:32.925161] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:157) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] submit log success(service_type_=0, last_id=1726035709537832000, limited_id=1726035709537832000) [2024-09-11 06:21:32.925172] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] submit log with lock success(service_type_=0, last_id=1726035709537832000, limited_id=1726035709537832000) [2024-09-11 06:21:32.925177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1726035692924286000, start_id=1726035692924286000, end_id=1726035692924286001) [2024-09-11 06:21:32.925551] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:32.925563] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:32.925572] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.926251] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926278] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926293] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:32.926304] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:32.926310] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:32.926318] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926334] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926346] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:32.926349] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.926352] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.926355] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:32.926360] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926370] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691286836, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8964844GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:32.926380] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:32.926383] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:32.926385] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:32.926387] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:32.926407] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035692926390), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:32.926415] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:32.926422] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:32.926426] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2318, ddl_arg=NULL) [2024-09-11 06:21:33.028592] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8420133) [2024-09-11 06:21:33.029418] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5551, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.029609] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035693029498000, start_id=1726035693029498000, end_id=1726035693029498001) [2024-09-11 06:21:33.030043] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.030057] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.030064] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.030730] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.030761] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691074922, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.030785] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.030796] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.030802] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.030807] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.030821] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.030828] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.030852] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693030831), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.030881] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.030888] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.143808] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=8304925) [2024-09-11 06:21:33.144617] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5552, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.144805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035693144168000, start_id=1726035693144168000, end_id=1726035693144168001) [2024-09-11 06:21:33.145273] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.145286] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.145296] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.145981] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.146011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.146029] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.146044] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.146050] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:33.146059] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.146068] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.146095] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693146075), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.146104] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.146112] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.248360] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:33.249153] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5553, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.249313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035693248438000, start_id=1726035693248438000, end_id=1726035693248438001) [2024-09-11 06:21:33.249731] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:33.249740] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.249748] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.249754] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.250434] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.250461] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.250476] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.250485] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.250491] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.250498] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035691302745, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.250508] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.250515] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.250543] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693250518), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.250552] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.250562] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.363681] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:33.365129] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.365144] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.365153] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.365826] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.365838] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.365862] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693365842), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.365871] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.365877] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.468797] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7979932) [2024-09-11 06:21:33.469601] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5555, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.469763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035693468779000, start_id=1726035693468779000, end_id=1726035693468779001) [2024-09-11 06:21:33.470198] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.470210] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.470219] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.470880] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.470913] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.470929] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.470951] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.470956] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.470964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.470976] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.470987] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:33.470990] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:33.470993] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.470999] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:33.471002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.471013] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.471023] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:33.471028] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:33.471031] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.471033] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:33.471055] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693471036), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.471063] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.471071] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.471077] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2283, ddl_arg=NULL) [2024-09-11 06:21:33.574035] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7874700) [2024-09-11 06:21:33.574816] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5556, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.575027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035693574961000, start_id=1726035693574961000, end_id=1726035693574961001) [2024-09-11 06:21:33.575476] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.575489] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.575501] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.576190] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576247] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.576258] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.576265] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.576270] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576283] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576295] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:33.576298] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:33.576301] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.576306] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:33.576309] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576318] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.576330] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:33.576333] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:33.576335] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.576339] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:33.576358] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693576342), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.576365] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.576374] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.576378] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2344, ddl_arg=NULL) [2024-09-11 06:21:33.678681] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7770075) [2024-09-11 06:21:33.679495] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5557, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.679695] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035693679111000, start_id=1726035693679111000, end_id=1726035693679111001) [2024-09-11 06:21:33.679763] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=42, avg_log_batch_cnt=1, total_group_log_size=33457, avg_group_log_size=796) [2024-09-11 06:21:33.679772] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=33618) [2024-09-11 06:21:33.680107] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.680118] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.680128] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.680820] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.680850] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.680865] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.680877] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.680881] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.680887] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.680900] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.680911] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:33.680913] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:33.680919] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.680922] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:33.680927] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.680937] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:33.680949] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.680976] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693680953), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.680985] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.680992] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.785079] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7663655) [2024-09-11 06:21:33.785933] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5558, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.786115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035693785434000, start_id=1726035693785434000, end_id=1726035693785434001) [2024-09-11 06:21:33.786614] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.786630] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.786637] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.787335] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.787367] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.787383] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.787398] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.787405] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.787410] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.787424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.787429] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.787457] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693787433), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.787466] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.787471] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:33.890168] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7558579) [2024-09-11 06:21:33.891048] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5559, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:33.891226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035693890560000, start_id=1726035693890560000, end_id=1726035693890560001) [2024-09-11 06:21:33.891713] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:33.891729] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:33.891736] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.892485] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.892516] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.892538] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:33.892548] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:33.892555] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:33.892563] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:33.892576] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:33.892586] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.892594] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:33.892618] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035693892597), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:33.892636] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:33.892643] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.098162] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7350570) [2024-09-11 06:21:34.099006] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5560, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.099189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035694098958000, start_id=1726035694098958000, end_id=1726035694098958001) [2024-09-11 06:21:34.099638] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.099652] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.099660] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.100303] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100333] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100349] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.100360] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.100366] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.100373] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100384] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100396] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.100398] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.100404] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.100407] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.100413] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100424] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.100435] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.100439] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.100441] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.100453] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.100480] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694100458), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.100489] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.100496] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.100500] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2340, ddl_arg=NULL) [2024-09-11 06:21:34.213587] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7235144) [2024-09-11 06:21:34.214436] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5561, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.214645] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035694213783000, start_id=1726035694213783000, end_id=1726035694213783001) [2024-09-11 06:21:34.215125] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.215141] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.215149] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.215886] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.215916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.215932] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.215944] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.215950] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.215956] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.215967] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.215979] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.215982] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.215987] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.215992] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.215995] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.216007] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.216017] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.216020] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.216023] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.216026] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.216050] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694216029), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.216059] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.216066] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.216072] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2487, ddl_arg=NULL) [2024-09-11 06:21:34.318576] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7130175) [2024-09-11 06:21:34.319384] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5562, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.319556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035694319224000, start_id=1726035694319224000, end_id=1726035694319224001) [2024-09-11 06:21:34.320026] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:34.320038] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.320046] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.320054] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.320725] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320759] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320781] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.320792] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.320796] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.320802] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320813] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320825] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.320827] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.320831] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.320837] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.320840] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.320863] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.320866] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.320868] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.320870] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.320896] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694320876), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.320905] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.320914] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.320918] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2344, ddl_arg=NULL) [2024-09-11 06:21:34.433738] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=7015007) [2024-09-11 06:21:34.434605] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5563, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.434803] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035694434068000, start_id=1726035694434068000, end_id=1726035694434068001) [2024-09-11 06:21:34.435216] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.435228] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.435237] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.435894] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.435924] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.435939] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.435951] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.435957] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.435964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.435979] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.435996] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.436000] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.436005] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.436012] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.436017] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.436030] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.436041] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.436044] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.436046] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.436049] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.436073] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694436052), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.436081] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.436088] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.436095] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2360, ddl_arg=NULL) [2024-09-11 06:21:34.538617] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6910113) [2024-09-11 06:21:34.539377] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5564, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.539539] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035694539309000, start_id=1726035694539309000, end_id=1726035694539309001) [2024-09-11 06:21:34.539980] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.539994] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.540003] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.540673] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540702] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540716] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.540727] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.540731] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.540739] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540762] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.540765] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.540769] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.540783] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.540787] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540797] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.540806] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.540808] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.540811] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.540814] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.540840] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694540818), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.540848] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.540857] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.540861] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2246, ddl_arg=NULL) [2024-09-11 06:21:34.643735] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6804994) [2024-09-11 06:21:34.644566] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5565, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.644741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035694644546000, start_id=1726035694644546000, end_id=1726035694644546001) [2024-09-11 06:21:34.645168] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.645181] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.645190] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.645848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645876] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645892] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.645909] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.645915] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.645922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645945] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.645948] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.645951] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.645955] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.645957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645970] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.645978] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.645981] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.645983] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.645985] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.646007] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694645988), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.646015] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.646021] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.646027] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2294, ddl_arg=NULL) [2024-09-11 06:21:34.748846] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6699895) [2024-09-11 06:21:34.749635] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5566, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.749820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1726035694749767000, start_id=1726035694749767000, end_id=1726035694749767001) [2024-09-11 06:21:34.749878] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=33, avg_log_batch_cnt=1, total_group_log_size=25847, avg_group_log_size=783) [2024-09-11 06:21:34.749889] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=25847) [2024-09-11 06:21:34.750243] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.750255] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.750263] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.750949] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.750979] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:34.751000] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.751009] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.751015] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.751042] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694751020), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.751051] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.751057] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.877976] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6570762) [2024-09-11 06:21:34.878821] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5567, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.878999] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035694878122000, start_id=1726035694878122000, end_id=1726035694878122001) [2024-09-11 06:21:34.879414] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.879429] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.879437] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.880171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:34.880213] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.880221] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.880249] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694880226), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.880258] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.880270] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.993673] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6455073) [2024-09-11 06:21:34.994608] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5568, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:34.994797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035694994018000, start_id=1726035694994018001, end_id=1726035694994018002) [2024-09-11 06:21:34.995231] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:34.995244] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:34.995254] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.995960] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.995991] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693075257, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.996005] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:34.996019] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:34.996025] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:34.996031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.996043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.996054] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:34.996058] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.996062] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.996065] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:34.996068] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.996079] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:34.996091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:34.996095] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:34.996098] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:34.996101] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:34.996131] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035694996106), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:34.996142] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:34.996149] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:34.996154] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2484, ddl_arg=NULL) [2024-09-11 06:21:35.098741] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6349994) [2024-09-11 06:21:35.099591] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5569, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.099826] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035695099200000, start_id=1726035695099200000, end_id=1726035695099200001) [2024-09-11 06:21:35.100290] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.100305] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.100313] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.101013] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101051] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101064] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.101075] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.101081] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.101089] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101100] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101112] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.101115] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.101121] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.101124] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.101130] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101140] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.101151] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.101153] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.101156] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.101158] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.101182] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695101162), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.101190] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.101197] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.101201] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2461, ddl_arg=NULL) [2024-09-11 06:21:35.203724] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6245006) [2024-09-11 06:21:35.204523] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5570, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.204703] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035695204597000, start_id=1726035695204597000, end_id=1726035695204597001) [2024-09-11 06:21:35.205232] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.205246] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.205255] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.205929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.205957] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.205974] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.205984] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.205991] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.205996] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.206010] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693303180, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.206022] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.206025] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.206029] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.206035] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.206039] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.206050] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035693287203, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.206060] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.206063] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.206070] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.206073] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.206099] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695206077), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.206107] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.206113] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.206120] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2398, ddl_arg=NULL) [2024-09-11 06:21:35.308697] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6140035) [2024-09-11 06:21:35.309522] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5571, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.309689] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035695308859000, start_id=1726035695308859000, end_id=1726035695308859001) [2024-09-11 06:21:35.310171] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.310185] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.310194] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.310855] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310882] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310899] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.310909] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.310915] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.310922] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310934] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310945] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.310948] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.310954] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.310957] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.310963] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310974] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.310983] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.310988] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.310991] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.310994] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.311018] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695310997), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.311027] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.311034] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.311039] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2344, ddl_arg=NULL) [2024-09-11 06:21:35.414213] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=6034525) [2024-09-11 06:21:35.415035] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5572, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.415205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035695414201000, start_id=1726035695414201000, end_id=1726035695414201001) [2024-09-11 06:21:35.415647] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:35.415657] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.415665] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.415671] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.416349] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416395] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.416406] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.416411] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.416416] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416428] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416438] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.416441] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.416447] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.416451] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.416455] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416465] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.416473] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.416476] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.416479] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.416482] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.416514] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695416485), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.416526] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.416536] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.416543] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2332, ddl_arg=NULL) [2024-09-11 06:21:35.519171] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=53] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5929564) [2024-09-11 06:21:35.519990] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5573, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.520181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:35.520647] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.520664] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.520675] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.521509] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.521548] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=38] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.521582] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.521593] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.521601] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.521639] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695521612), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.521650] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.521661] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.623978] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5824755) [2024-09-11 06:21:35.624733] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5574, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.624917] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035695624540000, start_id=1726035695624540000, end_id=1726035695624540001) [2024-09-11 06:21:35.625354] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.625367] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.625375] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.626066] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626104] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=37] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626128] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=23] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.626143] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.626152] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.626162] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626202] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.626207] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.626215] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.626220] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.626227] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626247] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.626273] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.626278] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.626285] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.626290] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.626319] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695626298), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.626330] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.626342] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:35.626351] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [DDL] execute ddl like stmt(ret=-4122, cost=2375, ddl_arg=NULL) [2024-09-11 06:21:35.728855] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:35.729719] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5575, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.730397] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.731166] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.731205] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.731216] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.731245] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695731225), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.731257] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.833980] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5614758) [2024-09-11 06:21:35.834767] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5576, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.834975] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1726035695834839000, start_id=1726035695834839000, end_id=1726035695834839001) [2024-09-11 06:21:35.835413] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:35.836100] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836128] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836143] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:35.836155] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:35.836162] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:35.836169] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836192] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:35.836194] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.836198] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.836204] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:35.836206] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836218] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.836227] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:35.836229] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:35.836232] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.836234] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:35.836257] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695836238), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.836266] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.836274] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2296, ddl_arg=NULL) [2024-09-11 06:21:35.938645] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5510081) [2024-09-11 06:21:35.939464] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5577, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:35.939656] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035695939213000, start_id=1726035695939213000, end_id=1726035695939213001) [2024-09-11 06:21:35.940135] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:35.940151] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:35.940159] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.940805] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:35.940846] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.940854] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:35.940877] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035695940857), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:35.940885] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:35.940893] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.053878] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5394846) [2024-09-11 06:21:36.054680] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5578, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.054867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035696053906000, start_id=1726035696053906000, end_id=1726035696053906001) [2024-09-11 06:21:36.055278] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.055291] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.055299] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.055972] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056001] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056017] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.056036] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.056042] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.056048] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056059] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056071] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.056074] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.056078] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.056084] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.056087] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056098] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.056108] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.056110] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.056112] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.056116] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.056141] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696056121), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.056150] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.056157] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.056162] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2287, ddl_arg=NULL) [2024-09-11 06:21:36.158770] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5289950) [2024-09-11 06:21:36.159501] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5579, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.159671] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035696159193000, start_id=1726035696159193000, end_id=1726035696159193001) [2024-09-11 06:21:36.160068] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.160081] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.160088] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.160945] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.160978] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.160994] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.161004] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.161010] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.161015] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.161027] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.161039] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.161042] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.161047] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.161056] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.161061] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.161077] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.161091] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.161094] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.161096] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.161099] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.161122] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696161105), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.161131] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.161138] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.161143] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2373, ddl_arg=NULL) [2024-09-11 06:21:36.263769] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5185069) [2024-09-11 06:21:36.264594] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5580, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.264768] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035696264583000, start_id=1726035696264583000, end_id=1726035696264583001) [2024-09-11 06:21:36.265207] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=40][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.265221] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.265231] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.265896] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.265926] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.265941] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.265954] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.265960] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.265967] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.265979] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.265988] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.265991] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.265996] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.265999] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.266005] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.266019] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.266030] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.266033] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.266035] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.266037] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.266062] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696266044), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.266071] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.266078] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.266083] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2316, ddl_arg=NULL) [2024-09-11 06:21:36.368628] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=5080105) [2024-09-11 06:21:36.369452] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5581, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.369627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035696368714000, start_id=1726035696368714000, end_id=1726035696368714001) [2024-09-11 06:21:36.370081] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.370094] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.370102] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.370750] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370811] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=60] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370826] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.370836] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.370842] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.370848] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370860] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370871] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.370874] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.370879] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.370882] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.370885] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370896] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.370906] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.370910] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.370912] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.370914] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.370939] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696370920), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.370947] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.370955] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.370962] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2335, ddl_arg=NULL) [2024-09-11 06:21:36.473610] INFO [RS] add_server (ob_root_service.cpp:7915) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4975115) [2024-09-11 06:21:36.474385] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5582, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.474544] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035696473942000, start_id=1726035696473942000, end_id=1726035696473942001) [2024-09-11 06:21:36.474986] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:36.475001] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.475010] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.475018] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.475648] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475677] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475693] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.475703] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.475709] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.475715] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475726] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475738] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.475741] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.475746] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.475751] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.475757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475782] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.475792] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.475794] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.475797] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.475801] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.475823] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696475805), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.475832] INFO [RS] add_server (ob_root_service.cpp:7928) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.475838] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.475844] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2236, ddl_arg=NULL) [2024-09-11 06:21:36.578450] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4870284) [2024-09-11 06:21:36.579257] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5583, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.579417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035696579216000, start_id=1726035696579216000, end_id=1726035696579216001) [2024-09-11 06:21:36.579891] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.579904] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.579912] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.580578] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580605] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580620] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.580632] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.580638] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.580642] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580653] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580662] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.580665] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.580668] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.580674] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.580676] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580688] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.580697] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.580700] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.580702] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.580704] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.580727] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696580710), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.580734] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.580741] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.580745] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2297, ddl_arg=NULL) [2024-09-11 06:21:36.683739] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4764997) [2024-09-11 06:21:36.684593] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5584, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.684802] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:36.685234] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.685249] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.685258] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.685935] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.685964] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.685988] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.685996] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.686021] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696686000), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.686030] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.686038] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.788746] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4659995) [2024-09-11 06:21:36.789862] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5585, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.790069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035696790003000, start_id=1726035696790003000, end_id=1726035696790003001) [2024-09-11 06:21:36.790136] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=33, avg_log_batch_cnt=1, total_group_log_size=26499, avg_group_log_size=803) [2024-09-11 06:21:36.790146] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=26310) [2024-09-11 06:21:36.790530] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.790540] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.790547] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.791263] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791291] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791306] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.791319] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.791324] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.791331] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791353] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.791356] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.791362] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.791365] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:36.791368] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791379] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.791388] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:36.791391] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.791393] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.791395] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:36.791415] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696791399), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.791423] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.791430] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:36.791434] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2690, ddl_arg=NULL) [2024-09-11 06:21:36.896930] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4551809) [2024-09-11 06:21:36.897898] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5586, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:36.898120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1726035696897335000, start_id=1726035696897335000, end_id=1726035696897335001) [2024-09-11 06:21:36.898646] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:36.898661] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:36.898670] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.899438] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.899470] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.899486] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:36.899497] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:36.899504] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:36.899511] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.899524] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:36.899536] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:36.899539] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:36.899546] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.899550] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:36.899557] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:36.899587] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035696899560), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:36.899597] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:36.899604] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.001799] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4446932) [2024-09-11 06:21:37.002640] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5587, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.002842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035697002561000, start_id=1726035697002561000, end_id=1726035697002561001) [2024-09-11 06:21:37.003273] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:37.003942] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.003973] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695075591, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.003989] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.004001] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.004007] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.004014] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.004026] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.004035] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.004037] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.004043] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.004046] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.004049] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.004060] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.004071] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.004073] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.004076] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.004078] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.004105] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697004082), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.004114] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.004123] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2326, ddl_arg=NULL) [2024-09-11 06:21:37.106594] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4342146) [2024-09-11 06:21:37.107462] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5588, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.107654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035697106838000, start_id=1726035697106838000, end_id=1726035697106838001) [2024-09-11 06:21:37.108112] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.108125] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.108140] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.108842] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108873] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108890] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.108902] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.108909] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.108916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108929] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108940] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.108943] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.108946] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.108951] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.108955] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108971] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.108980] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.108982] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.108985] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.108988] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.109012] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697108992), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.109018] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.109025] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.109036] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] [DDL] execute ddl like stmt(ret=-4122, cost=2444, ddl_arg=NULL) [2024-09-11 06:21:37.211470] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4237254) [2024-09-11 06:21:37.212325] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5589, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.212521] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1726035697211967000, start_id=1726035697211967000, end_id=1726035697211967001) [2024-09-11 06:21:37.212975] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.212990] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.212998] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.213680] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213711] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213726] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.213739] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.213744] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.213748] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213761] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695303560, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213772] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.213805] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.213808] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.213814] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.213817] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213828] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035695287555, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9121094GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.213837] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.213839] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.213841] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.213843] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.213869] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697213848), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.213877] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.213885] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.213889] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2422, ddl_arg=NULL) [2024-09-11 06:21:37.316472] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4132253) [2024-09-11 06:21:37.317262] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5590, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.317429] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035697317376000, start_id=1726035697317376000, end_id=1726035697317376001) [2024-09-11 06:21:37.317834] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.317851] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.317859] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.318511] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.318540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.318555] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.318568] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:37.318574] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.318580] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.318591] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.318614] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697318594), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.318621] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.318626] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.421178] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=4027543) [2024-09-11 06:21:37.422055] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5591, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.422239] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035697421512000, start_id=1726035697421512000, end_id=1726035697421512001) [2024-09-11 06:21:37.422680] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.422693] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.422701] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.423376] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423406] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423422] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.423434] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.423440] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.423447] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423459] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423470] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.423473] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.423478] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.423481] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.423486] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423496] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.423506] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.423509] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.423511] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.423513] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.423541] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697423519), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.423552] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.423560] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.423564] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2388, ddl_arg=NULL) [2024-09-11 06:21:37.525809] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3922934) [2024-09-11 06:21:37.526587] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5592, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.526756] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035697525744000, start_id=1726035697525744000, end_id=1726035697525744001) [2024-09-11 06:21:37.527197] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:37.527206] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.527215] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.527222] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.527890] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.527916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.527930] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.527940] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.527946] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.527951] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.527962] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.527973] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.527976] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.527980] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.527986] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.527990] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.528002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.528011] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.528013] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.528016] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.528019] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.528042] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697528022), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.528052] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.528064] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.528069] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] [DDL] execute ddl like stmt(ret=-4122, cost=2262, ddl_arg=NULL) [2024-09-11 06:21:37.630453] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3818279) [2024-09-11 06:21:37.631207] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5593, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.631381] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035697630942000, start_id=1726035697630942000, end_id=1726035697630942001) [2024-09-11 06:21:37.631871] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.631885] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.631891] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.632530] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632559] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632577] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.632591] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.632597] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.632604] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632616] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632633] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.632636] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.632639] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.632643] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.632647] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632659] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.632668] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.632671] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.632673] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.632676] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.632695] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697632679), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.632703] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.632710] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.632714] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2263, ddl_arg=NULL) [2024-09-11 06:21:37.735098] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3713642) [2024-09-11 06:21:37.736512] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.736527] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.736538] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.737254] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.737265] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.737287] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697737269), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.737297] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.737306] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.839731] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3609002) [2024-09-11 06:21:37.840528] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5595, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.840709] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035697840479000, start_id=1726035697840479000, end_id=1726035697840479001) [2024-09-11 06:21:37.841176] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.841194] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.841202] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.841883] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.841916] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=32] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.841931] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.841944] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.841951] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.841959] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.841971] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.841982] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.841986] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.841989] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.841993] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.841996] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.842006] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.842016] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.842019] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.842021] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.842024] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.842045] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697842027), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.842053] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.842062] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.842065] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2336, ddl_arg=NULL) [2024-09-11 06:21:37.944387] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3504341) [2024-09-11 06:21:37.945470] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5596, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:37.945731] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035697945714000, start_id=1726035697945714000, end_id=1726035697945714001) [2024-09-11 06:21:37.946234] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:37.946254] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:37.946266] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.947226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947253] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947274] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:37.947289] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:37.947298] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:37.947304] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947329] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:37.947355] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.947363] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.947369] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:37.947374] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947394] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:37.947412] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:37.947417] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:37.947424] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:37.947428] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:37.947459] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035697947436), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:37.947470] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:37.947482] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:37.947491] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] [DDL] execute ddl like stmt(ret=-4122, cost=3105, ddl_arg=NULL) [2024-09-11 06:21:38.049173] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3399556) [2024-09-11 06:21:38.049952] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5597, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.050110] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035698049789000, start_id=1726035698049789000, end_id=1726035698049789001) [2024-09-11 06:21:38.050506] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.050519] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:38.051221] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051247] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051263] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.051273] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.051281] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.051287] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051303] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051319] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.051323] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.051330] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.051335] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.051342] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051358] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.051373] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.051377] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.051381] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.051388] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.051429] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698051393), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.051440] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.051451] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] [DDL] execute ddl like stmt(ret=-4122, cost=2280, ddl_arg=NULL) [2024-09-11 06:21:38.153646] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3295087) [2024-09-11 06:21:38.154437] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5598, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.154601] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1726035698153649000, start_id=1726035698153649000, end_id=1726035698153649001) [2024-09-11 06:21:38.155045] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.155059] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.155068] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.155709] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.155749] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.155757] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.155771] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.155815] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698155798), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.155824] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.155832] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.258093] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3190636) [2024-09-11 06:21:38.258896] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5599, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.259062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035698258780000, start_id=1726035698258780000, end_id=1726035698258780001) [2024-09-11 06:21:38.259484] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.259497] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.259505] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.260197] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260226] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260241] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.260251] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.260257] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.260262] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260273] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260285] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.260289] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.260295] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.260299] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.260301] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260313] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.260325] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.260328] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.260331] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.260334] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.260357] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698260340), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.260365] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.260371] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.260389] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] [DDL] execute ddl like stmt(ret=-4122, cost=2297, ddl_arg=NULL) [2024-09-11 06:21:38.362684] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=3086046) [2024-09-11 06:21:38.363524] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5600, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.363706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1726035698362938000, start_id=1726035698362938000, end_id=1726035698362938001) [2024-09-11 06:21:38.364143] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.364158] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.364167] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.364852] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:38.364890] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.364897] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.364921] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698364900), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.364931] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.364938] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.467389] INFO [RS] add_server (ob_root_service.cpp:7915) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2981353) [2024-09-11 06:21:38.468202] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5601, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.468382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035698468230000, start_id=1726035698468230000, end_id=1726035698468230001) [2024-09-11 06:21:38.468785] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.468797] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.468805] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.469465] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469495] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469515] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.469528] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.469533] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.469540] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469558] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469572] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.469578] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.469583] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.469589] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.469595] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469610] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.469626] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.469631] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.469636] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.469641] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.469667] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698469647), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.469674] INFO [RS] add_server (ob_root_service.cpp:7928) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.469683] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.469691] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55191][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] [DDL] execute ddl like stmt(ret=-4122, cost=2302, ddl_arg=NULL) [2024-09-11 06:21:38.572121] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2876608) [2024-09-11 06:21:38.572919] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5602, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.573084] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035698572329000, start_id=1726035698572329000, end_id=1726035698572329001) [2024-09-11 06:21:38.573507] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:38.573516] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.573524] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.573531] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.574182] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574212] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574227] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.574241] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.574249] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.574261] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574274] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574285] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.574288] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.574291] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.574297] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.574300] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574312] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.574322] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.574325] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.574327] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.574330] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.574415] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698574333), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.574425] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.574433] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.574436] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2317, ddl_arg=NULL) [2024-09-11 06:21:38.676994] INFO [RS] add_server (ob_root_service.cpp:7915) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2771724) [2024-09-11 06:21:38.677822] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5603, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.678008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1726035698677600000, start_id=1726035698677600000, end_id=1726035698677600001) [2024-09-11 06:21:38.678347] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.678360] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.678369] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.679028] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679059] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679072] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.679086] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.679093] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.679103] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679121] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679137] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.679141] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.679149] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.679162] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.679166] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679178] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.679190] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:38.679194] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.679216] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698679198), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.679225] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.679230] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.781726] INFO [RS] add_server (ob_root_service.cpp:7915) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2667004) [2024-09-11 06:21:38.782498] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5604, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.782668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035698781781000, start_id=1726035698781781000, end_id=1726035698781781001) [2024-09-11 06:21:38.783080] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.783094] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.783102] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.783763] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783816] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=53] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783832] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.783842] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.783849] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.783854] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783865] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783877] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.783880] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.783885] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.783891] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.783894] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783904] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.783915] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.783917] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.783920] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.783923] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.783946] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698783926), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.783954] INFO [RS] add_server (ob_root_service.cpp:7928) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.783960] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.783966] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55220][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2242, ddl_arg=NULL) [2024-09-11 06:21:38.887734] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5605, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.887938] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035698887090000, start_id=1726035698887090000, end_id=1726035698887090001) [2024-09-11 06:21:38.888318] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.888330] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.888338] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.889011] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.889039] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.889053] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.889065] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.889071] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.889078] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.889097] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.889103] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.889126] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698889108), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.889133] INFO [RS] add_server (ob_root_service.cpp:7928) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.889140] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55219][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.991953] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2456806) [2024-09-11 06:21:38.992746] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5606, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:38.992934] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035698992287000, start_id=1726035698992287000, end_id=1726035698992287001) [2024-09-11 06:21:38.993338] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:38.993351] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:38.993363] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.994043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994076] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=33] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697075931, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994095] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:38.994106] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:38.994115] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:38.994124] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994141] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994156] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:38.994159] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.994163] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.994168] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:38.994172] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994184] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697287986, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0058594GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:38.994194] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:38.994197] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:38.994199] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:38.994202] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:38.994229] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035698994208), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:38.994237] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:38.994244] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:38.994251] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2301, ddl_arg=NULL) [2024-09-11 06:21:39.095095] INFO [RS] add_server (ob_root_service.cpp:7915) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2353628) [2024-09-11 06:21:39.095964] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5607, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.096136] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035699095440000, start_id=1726035699095440000, end_id=1726035699095440001) [2024-09-11 06:21:39.096532] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.096544] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.096553] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.097256] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.097286] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.097302] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.097314] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.097320] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.097327] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.097339] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035697303991, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.097350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:39.097353] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.097357] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.097363] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:39.097368] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.097396] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699097375), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.097407] INFO [RS] add_server (ob_root_service.cpp:7928) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.097416] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55214][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.200081] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2248643) [2024-09-11 06:21:39.200936] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5608, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.201117] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035699200557000, start_id=1726035699200557000, end_id=1726035699200557001) [2024-09-11 06:21:39.202222] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.202261] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.202269] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.202296] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699202273), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.202307] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.305148] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2143588) [2024-09-11 06:21:39.306003] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5609, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.306183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035699305835000, start_id=1726035699305835000, end_id=1726035699305835001) [2024-09-11 06:21:39.306600] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.306614] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.306622] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.307277] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307307] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307323] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.307334] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.307343] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.307352] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307364] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307375] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:39.307377] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.307381] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.307386] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:39.307390] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307401] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.307412] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:39.307415] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.307417] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.307420] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:39.307446] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699307424), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.307455] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.307463] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.307469] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2323, ddl_arg=NULL) [2024-09-11 06:21:39.410530] INFO [RS] add_server (ob_root_service.cpp:7915) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=2038204) [2024-09-11 06:21:39.411394] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5610, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.411564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1726035699411127000, start_id=1726035699411127000, end_id=1726035699411127001) [2024-09-11 06:21:39.411984] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.411995] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.412001] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.412612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.412636] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=24] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:39.412650] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.412663] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.412667] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.412687] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699412670), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.412693] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.412698] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.515613] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1933112) [2024-09-11 06:21:39.516430] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5611, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.516605] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035699516289000, start_id=1726035699516289000, end_id=1726035699516289001) [2024-09-11 06:21:39.517010] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.517022] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.517031] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.517685] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.517713] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.517734] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=19] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.517754] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.517759] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.517768] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.517791] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=22] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.517805] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:39.517811] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.517818] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.517847] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699517826), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.517855] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.517864] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.620666] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1828060) [2024-09-11 06:21:39.621544] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:39.621760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035699621364000, start_id=1726035699621364000, end_id=1726035699621364001) [2024-09-11 06:21:39.622249] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:39.622259] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.622274] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.622281] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.622989] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.623019] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.623035] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.623045] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.623052] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.623058] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.623072] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.623078] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.623164] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699623085), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.623173] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.623180] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.725929] INFO [RS] add_server (ob_root_service.cpp:7915) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1722810) [2024-09-11 06:21:39.726684] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5613, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.726897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035699726523000, start_id=1726035699726523000, end_id=1726035699726523001) [2024-09-11 06:21:39.727305] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.727317] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.727326] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.727980] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728008] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728024] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.728036] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.728042] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.728047] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728059] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728070] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:39.728073] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.728077] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.728081] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:39.728085] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728097] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.728110] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:39.728112] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.728115] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.728119] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:39.728143] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699728126), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.728151] INFO [RS] add_server (ob_root_service.cpp:7928) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.728158] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.728165] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55216][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] [DDL] execute ddl like stmt(ret=-4122, cost=2238, ddl_arg=NULL) [2024-09-11 06:21:39.831131] INFO [RS] add_server (ob_root_service.cpp:7915) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1617605) [2024-09-11 06:21:39.831961] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5614, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.832152] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035699831795000, start_id=1726035699831795000, end_id=1726035699831795001) [2024-09-11 06:21:39.832576] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.832590] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.832597] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.833255] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833285] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833299] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.833314] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.833321] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.833326] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833338] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833350] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:39.833352] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:39.833357] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.833360] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:39.833363] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833372] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.833384] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.833409] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699833387), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.833426] INFO [RS] add_server (ob_root_service.cpp:7928) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.833434] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55224][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:39.936516] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:39.937339] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5615, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:39.937537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035699937070000, start_id=1726035699937070000, end_id=1726035699937070001) [2024-09-11 06:21:39.937605] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.21.8.26:2882", role="LEADER", total_group_log_cnt=36, avg_log_batch_cnt=1, total_group_log_size=28804, avg_group_log_size=800) [2024-09-11 06:21:39.937616] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.21.8.26:2882", has_set_deleted:false}, append size=28804) [2024-09-11 06:21:39.937948] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:39.937960] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:39.937967] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.938632] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.938661] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.938677] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:39.938689] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:39.938695] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:39.938702] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:39.938716] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.938723] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:39.938746] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035699938727), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:39.938755] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:39.938762] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.041793] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1406944) [2024-09-11 06:21:40.042549] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5616, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.042734] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035700042237000, start_id=1726035700042237000, end_id=1726035700042237001) [2024-09-11 06:21:40.043215] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.043229] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.043238] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.043923] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.043953] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.043967] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.043978] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.043984] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.043989] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.044002] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.044014] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:40.044017] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.044021] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.044026] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:40.044031] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.044043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.044053] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:40.044056] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.044058] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.044062] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:40.044085] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700044068), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.044094] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.044101] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.044112] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] [DDL] execute ddl like stmt(ret=-4122, cost=2320, ddl_arg=NULL) [2024-09-11 06:21:40.146973] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1301754) [2024-09-11 06:21:40.147734] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5617, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.147942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035700147488000, start_id=1726035700147488000, end_id=1726035700147488001) [2024-09-11 06:21:40.148369] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.148382] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.148394] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.149094] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.149122] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.149137] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.149149] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.149156] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.149161] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.149171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.149184] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:40.149187] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.149192] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.149213] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700149196), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.149220] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.149228] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.253030] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1726035700252706000, start_id=1726035700252706000, end_id=1726035700252706001) [2024-09-11 06:21:40.253472] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.253486] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.253495] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.254151] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.254178] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.254195] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.254205] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.254218] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.254222] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:40.254230] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.254251] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700254234), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.254259] INFO [RS] add_server (ob_root_service.cpp:7928) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.254265] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55213][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.357368] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=1091368) [2024-09-11 06:21:40.358126] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5619, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.358305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035700357833000, start_id=1726035700357833000, end_id=1726035700357833001) [2024-09-11 06:21:40.358655] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.358668] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.358677] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.359374] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.359401] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=27] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.359417] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.359428] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:40.359437] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.359442] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.359462] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700359445), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.359470] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.359476] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.463033] INFO [RS] add_server (ob_root_service.cpp:7915) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=985693) [2024-09-11 06:21:40.463830] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5620, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.464011] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035700463083000, start_id=1726035700463083000, end_id=1726035700463083001) [2024-09-11 06:21:40.464452] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.464464] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.464472] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.465144] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465171] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=26] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465186] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.465197] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.465204] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.465213] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465230] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465240] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:40.465243] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.465246] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.465249] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:40.465255] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465267] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.465275] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:40.465278] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.465280] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.465283] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:40.465307] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700465290), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.465314] INFO [RS] add_server (ob_root_service.cpp:7928) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.465322] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.465325] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55215][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2294, ddl_arg=NULL) [2024-09-11 06:21:40.567901] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=880830) [2024-09-11 06:21:40.568650] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5621, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.568835] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1726035700568296000, start_id=1726035700568296000, end_id=1726035700568296001) [2024-09-11 06:21:40.569228] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.569241] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.569249] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.569901] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.569931] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.569946] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.569958] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.569971] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.569975] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.569988] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.569997] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:40.570000] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.570003] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.570005] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:40.570008] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.570020] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.570029] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:40.570031] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.570033] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.570036] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:40.570058] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700570041), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.570066] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.570073] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.664151] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=784593) [2024-09-11 06:21:40.664983] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5622, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.665151] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1726035700665026000, start_id=1726035700665026000, end_id=1726035700665026001) [2024-09-11 06:21:40.665532] WDIAG [RPC] check_blacklist (ob_poc_rpc_proxy.cpp:270) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15][errcode=-4122] address in blacklist(ret=-4122, addr="10.21.8.23:2882") [2024-09-11 06:21:40.665542] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.665550] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.665558] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.666257] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.666292] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=34] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.666307] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.666318] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.666324] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:40.666332] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.666338] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.666370] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700666342), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.666378] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.666390] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.768965] INFO [RS] add_server (ob_root_service.cpp:7915) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=679767) [2024-09-11 06:21:40.769812] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=20] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5623, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.769996] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035700769066000, start_id=1726035700769066000, end_id=1726035700769066001) [2024-09-11 06:21:40.770371] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.770388] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.770397] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.771096] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.771132] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=35] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.771148] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.771162] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.771170] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.771196] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700771174), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.771206] INFO [RS] add_server (ob_root_service.cpp:7928) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.771215] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55223][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.875148] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=573579) [2024-09-11 06:21:40.875985] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5624, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.876183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1726035700875178000, start_id=1726035700875178000, end_id=1726035700875178001) [2024-09-11 06:21:40.876624] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.876639] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.876647] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.877337] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.877369] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=31] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.877389] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.877400] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.877407] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.877412] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.877423] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.877434] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:40.877438] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.877445] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.877471] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700877448), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.877481] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.877487] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.979908] INFO [RS] add_server (ob_root_service.cpp:7915) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=468814) [2024-09-11 06:21:40.980704] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5625, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:40.980881] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035700980312000, start_id=1726035700980312000, end_id=1726035700980312001) [2024-09-11 06:21:40.981291] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:40.981304] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:40.981312] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.981974] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982003] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=28] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699076256, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982019] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:40.982031] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] added async task to update rslist(force_update=false) [2024-09-11 06:21:40.982037] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:40.982043] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982055] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982066] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:40.982069] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.982072] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.982078] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:40.982080] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982091] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:40.982101] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:40.982104] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:40.982106] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:40.982108] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:40.982135] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035700982114), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:40.982143] INFO [RS] add_server (ob_root_service.cpp:7928) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:40.982150] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:40.982154] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55221][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2248, ddl_arg=NULL) [2024-09-11 06:21:41.084540] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=364183) [2024-09-11 06:21:41.085368] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5626, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:41.085532] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035701085195000, start_id=1726035701085195000, end_id=1726035701085195001) [2024-09-11 06:21:41.085947] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:41.085959] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:41.085968] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.086613] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086643] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086659] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:41.086671] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] added async task to update rslist(force_update=false) [2024-09-11 06:21:41.086677] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:41.086684] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086695] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086707] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:41.086709] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.086715] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.086718] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:41.086722] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086733] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.086743] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:41.086745] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.086747] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.086750] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:41.086790] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035701086757), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:41.086801] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.086809] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:41.086814] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4122, cost=2275, ddl_arg=NULL) [2024-09-11 06:21:41.189398] INFO [RS] add_server (ob_root_service.cpp:7915) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=259328) [2024-09-11 06:21:41.190206] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5627, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:41.190407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1726035701189406000, start_id=1726035701189406000, end_id=1726035701189406001) [2024-09-11 06:21:41.190819] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:41.190833] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:41.190843] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.191582] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191612] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=30] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191631] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=18] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:41.191642] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] added async task to update rslist(force_update=false) [2024-09-11 06:21:41.191645] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:41.191649] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191662] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191673] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:41.191675] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.191682] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.191685] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:41.191688] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191697] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699288346, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:41.0917969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.191708] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:41.191710] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.191712] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.191714] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:41.191737] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035701191718), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:41.191744] INFO [RS] add_server (ob_root_service.cpp:7928) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.191749] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:41.191753] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55222][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2357, ddl_arg=NULL) [2024-09-11 06:21:41.294302] INFO [RS] add_server (ob_root_service.cpp:7915) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=17] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=154426) [2024-09-11 06:21:41.295143] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] fetch_new_max_id(ret=0, ret="OB_SUCCESS", size=1, tenant_id=1, fetch_id=5628, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:41.295327] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:390) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1726035701294747000, start_id=1726035701294747000, end_id=1726035701294747001) [2024-09-11 06:21:41.295760] WDIAG [RPC] send (ob_poc_rpc_proxy.h:147) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4122] check_blacklist failed(ret=-4122) [2024-09-11 06:21:41.295780] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:148) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4122] fail to check whether the server is empty(ret=-4122, ret="OB_RPC_POST_ERROR", addr="10.21.8.23:2882") [2024-09-11 06:21:41.295789] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4122] fail to add servers(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.296436] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296465] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=29] update server admin status, after update(server="10.21.8.26:2882", status={server:"10.21.8.26:2882", id:1, zone:"zone1", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701076586, block_migrate_in_time:0, stop_time:0, start_service_time:1725879801437334, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:true, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.7128906GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025798, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296479] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] self is RS and self status change, submit update rslist task(server="10.21.8.26:2882") [2024-09-11 06:21:41.296491] INFO [RS] submit_update_rslist_task (ob_root_service.cpp:1677) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] added async task to update rslist(force_update=false) [2024-09-11 06:21:41.296498] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.26:2882") [2024-09-11 06:21:41.296506] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] update server admin status, before update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296517] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] update server admin status, after update(server="10.21.8.27:2882", status={server:"10.21.8.27:2882", id:2, zone:"zone2", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035699304398, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806557627, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.8457031GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025803, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296528] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11] self is RS and self status change, submit update rslist task(server="10.21.8.27:2882") [2024-09-11 06:21:41.296531] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.296535] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.296540] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.27:2882") [2024-09-11 06:21:41.296543] INFO [RS] load_server_statuses (ob_server_manager.cpp:1463) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update server admin status, before update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701288752, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9042969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296555] INFO [RS] load_server_statuses (ob_server_manager.cpp:1474) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=12] update server admin status, after update(server="10.21.8.28:2882", status={server:"10.21.8.28:2882", id:3, zone:"zone3", build_version:"4.3.1.0_100000032024051615-bad90e897a7f0f56b8ce5c43e186aa8f4bab03c4(May 16 2024 17:21:43)", sql_port:2881, register_time:0, last_hb_time:1726035701288752, block_migrate_in_time:0, stop_time:0, start_service_time:1725879806481863, last_offline_time:0, last_server_behind_time:0, last_round_trip_time:0, admin_status:"NORMAL", hb_status:"lease_expired", with_rootserver:false, with_partition:true, resource_info:{cpu_capacity:72, cpu_assigned:33.5, cpu_assigned_max:69, mem_capacity:"224GB", mem_assigned:"189GB", mem_in_use:0GB, log_disk_capacity:3132.1875GB, log_disk_assigned:2779GB, data_disk_capacity:1565.64648GB, data_disk_in_use:40.9042969GB}, leader_cnt:-1, server_report_status:0, lease_expire_time:1725879840025806, ssl_key_expired_time:0, in_recovery_for_takenover_by_rs:false}) [2024-09-11 06:21:41.296565] INFO [RS] submit_update_all_server_task (ob_root_service.cpp:1376) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] self is RS and self status change, submit update rslist task(server="10.21.8.28:2882") [2024-09-11 06:21:41.296568] INFO [RS] try_lock (ob_update_rs_list_task.cpp:54) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] update rslist task exist, do not submit again(cnt=2) [2024-09-11 06:21:41.296570] WDIAG [RS] submit_update_rslist_task (ob_root_service.cpp:1683) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=0] fail to submit update rslist task, need retry(force_update=false) [2024-09-11 06:21:41.296572] INFO [RS] on_server_status_change (ob_root_service.cpp:189) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2] on_server_status_change finish(ret=0, ret="OB_SUCCESS", server="10.21.8.28:2882") [2024-09-11 06:21:41.296599] INFO [SHARE] add_event (ob_event_history_table_operator.h:261) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_rootservice_event_history", sql=INSERT INTO __all_rootservice_event_history (gmt_create, module, event, name1, value1, name2, value2, value3, value4, value5, value6, rs_svr_ip, rs_svr_port) VALUES (usec_to_time(1726035701296578), 'server', 'load_servers', 'ret', 0, 'has_build', 1, '', '', '', '', '10.21.8.26', 2882)) [2024-09-11 06:21:41.296610] INFO [RS] add_server (ob_root_service.cpp:7928) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=10] add server(ret=-4122, ret="OB_RPC_POST_ERROR", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.296617] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4122] process failed(ret=-4122) [2024-09-11 06:21:41.296620] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55218][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] [DDL] execute ddl like stmt(ret=-4122, cost=2320, ddl_arg=NULL) [2024-09-11 06:21:41.398944] INFO [RS] add_server (ob_root_service.cpp:7915) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5] sys tenant data version >= 4.2, add_server(arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}, timeout_ts=49788) [2024-09-11 06:21:41.399728] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:1985) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4012] table scan timeout(ret=-4012, current_time=1726035701398850, table_scan_param_timeout=1726035701348730, lbt()="0x17a30451 0x127e698a 0x5c8a9da 0x5c890b6 0x5c87bc0 0x5c869b0 0x5c8577e 0x5c8365f 0x5e5d615 0x5e5b3b2 0x5e3b377 0xfdbde6f 0x5e714c4 0x5e3b377 0xd81ac0e 0x5e3b377 0x609b40a 0x5fb4ce5 0x5fb391b 0x5fadb02 0x5faae03 0x5faa919 0x5faa612 0x17af7d5e 0x5ebf391 0x14ff8530 0x14ff9cee 0x8eef528 0xa1684a3 0xbae8187 0x6153c22 0x5c26066 0x5c1f72a 0xb3c3824 0x17b1814e 0x7f38570081cf 0x7f3856c39d83") [2024-09-11 06:21:41.399742] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:1999) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4012] failed to do table scan(ret=-4012, param={tablet_id:{id:121}, ls_id:{id:1}, column_ids:[18, 19, 20, 22], index_id:121, key_ranges:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], ss_key_ranges:[{"range":"table_id:121,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348730, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1725879596562040, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac818560, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2024-09-11 06:21:41.399901] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:653) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=158][errcode=-4012] failed to do table scan(ret=-4012, &iter=0x7f380d8a71d0, param={tablet_id:{id:121}, ls_id:{id:1}, column_ids:[18, 19, 20, 22], index_id:121, key_ranges:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], ss_key_ranges:[{"range":"table_id:121,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348730, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1725879596562040, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac818560, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}) [2024-09-11 06:21:41.399975] WDIAG [STORAGE] table_scan (ob_access_service.cpp:334) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=73][errcode=-4012] Fail to scan table, (ret=-4012, ls_id={id:1}, param={tablet_id:{id:121}, ls_id:{id:1}, column_ids:[18, 19, 20, 22], index_id:121, key_ranges:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], ss_key_ranges:[{"range":"table_id:121,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348730, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1725879596562040, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac818560, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}) [2024-09-11 06:21:41.400052] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_op.cpp:449) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=76][errcode=-4012] fail to scan table(scan_param_={tablet_id:{id:121}, ls_id:{id:1}, column_ids:[18, 19, 20, 22], index_id:121, key_ranges:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], ss_key_ranges:[{"range":"table_id:121,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348730, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1725879596562040, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac818560, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, ret=-4012) [2024-09-11 06:21:41.400129] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:336) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=76][errcode=-4012] fail to scan table(scan_param_={tablet_id:{id:121}, ls_id:{id:1}, column_ids:[18, 19, 20, 22], index_id:121, key_ranges:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], ss_key_ranges:[{"range":"table_id:121,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348730, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1725879596562040, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac818560, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, ret=-4012) [2024-09-11 06:21:41.400196] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:314) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=66][errcode=-4012] open das task op failed(ret=-4012) [2024-09-11 06:21:41.400201] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:98) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] start das task failed(ret=-4012, *task_wrapper.at(i)={parent:{tenant_id:1, task_id:175055656, op_type:1, errcode:-4012, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f2f01c853f0, tx_id:{txid:2961985}, state:2, addr:"10.21.8.26:2882", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1726035701398850, active_ts:1726035701398850, commit_ts:-1, finish_ts:-1, timeout_us:49254, lock_timeout_us:-1, expire_ts:1726035701448104, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1723810551, cluster_version:17180066048, seq_base:1726035701401469, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7f37e7dcc508, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:2961985}, scn:{branch:0, seq:2}}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, tablet_id:{id:121}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:121}, ls_id:{id:1}, server:"10.21.8.26:2882", in_retry:0, partition_id:121, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:121, access_column_ids:[cnt:4, 18, 19, 20, 22], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1725879596562040, table_param:{table_id:121, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:8, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, seq_read_column_count:3, max_col_index:8, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 2, 8], data:0x7f37ac0d8480, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 2, 6], data:0x7f37ac0d84a0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f37ac0d8440, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f37ac0d84f0, count:4, allocator:0x7f37c375e6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, pd_expr_spec:{calc_exprs:[cnt:0], access_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:0, pushdown_filters:[cnt:0], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:81, res_buf_off:711, dyn_buf_header_offset:695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0158}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:162, res_buf_off:735, dyn_buf_header_offset:719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0230}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:243, res_buf_off:879, dyn_buf_header_offset:863, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0308}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:0, res_buf_off:583, dyn_buf_header_offset:567, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f37c36f0080}], is_get:true, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:1726022380899392, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1726035701348730, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}}, scan_range:[{"range":"table_id:121,group_idx:0,[{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"};{"BIGINT":0},{"VARCHAR":"", collation:"utf8mb4_general_ci"},{"VARCHAR":"ob_max_used_server_id", collation:"utf8mb4_general_ci"}]"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}}) [2024-09-11 06:21:41.400333] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:268) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=131][errcode=-4012] failed to execute aggregated das task(ret=-4012, ret="OB_TIMEOUT", aggregated_task={server:"10.21.8.26:2882", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true) [2024-09-11 06:21:41.400342] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:251) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8][errcode=-4012] failed to execute all das task(ret=-4012) [2024-09-11 06:21:41.400348] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2386) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] execute all das scan task failed(ret=-4012) [2024-09-11 06:21:41.400355] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1604) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] fail to do table scan(ret=-4012) [2024-09-11 06:21:41.400357] WDIAG [SQL.ENG] inner_get_next_row_for_tsc (ob_table_scan_op.cpp:2163) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] failed to init before get row(ret=-4012) [2024-09-11 06:21:41.400363] WDIAG [SQL.ENG] inner_get_next_row_implement (ob_table_scan_op.cpp:2151) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] failed to get next row(ret=-4012) [2024-09-11 06:21:41.400366] WDIAG [SQL.ENG] inner_get_next_row (ob_table_scan_op.cpp:3125) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] get next row failed(ret=-4012) [2024-09-11 06:21:41.400373] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1171) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] inner get next row failed(ret=-4012, type=3, op="PHY_TABLE_SCAN", op_id=2) [2024-09-11 06:21:41.400387] WDIAG [SQL.ENG] get_next_row_from_child (ob_table_modify_op.cpp:1154) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=14][errcode=-4012] fail to get next row(ret=-4012) [2024-09-11 06:21:41.400394] WDIAG [SQL.ENG] inner_get_next_row (ob_table_lock_op.cpp:191) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] fail to get next row(ret=-4012) [2024-09-11 06:21:41.400398] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1171) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] inner get next row failed(ret=-4012, type=92, op="PHY_LOCK", op_id=1) [2024-09-11 06:21:41.400403] WDIAG [SQL.ENG] get_all_row_from_child (ob_material_op.cpp:84) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] fail to get next row(ret=-4012) [2024-09-11 06:21:41.400406] WDIAG [SQL.ENG] inner_get_next_row (ob_material_op.cpp:185) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] failed to get all row from child(child_=0x7f16ac824058, ret=-4012) [2024-09-11 06:21:41.400412] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1171) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] inner get next row failed(ret=-4012, type=34, op="PHY_MATERIAL", op_id=0) [2024-09-11 06:21:41.400416] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] get next row from operator failed(ret=-4012) [2024-09-11 06:21:41.400419] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:466) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] get next row from exec result failed(ret=-4012) [2024-09-11 06:21:41.400424] WDIAG [SERVER] open (ob_inner_sql_result.cpp:165) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] get_next_row failed(ret=-4012, has_tenant_resource()=true) [2024-09-11 06:21:41.400428] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:706) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] result set open failed(ret=-4012, executor={ObIExecutor:, sql:"SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE"}) [2024-09-11 06:21:41.400436] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:841) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] execute failed(ret=-4012, tenant_id=1, executor={ObIExecutor:, sql:"SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE"}, retry_cnt=0, local_sys_schema_version=1726022380899392, local_tenant_schema_version=1726022380899392) [2024-09-11 06:21:41.400444] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:986) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4012, err_:"OB_TIMEOUT", retry_type:0, client_ret:-4012}, need_retry=false) [2024-09-11 06:21:41.400464] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=8] REACH SYSLOG RATE LIMIT [bandwidth] [2024-09-11 06:21:41.400486] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=0][errcode=-4012] result set close failed(ret=-4012) [2024-09-11 06:21:41.400491] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] result set close failed(ret=-4012) [2024-09-11 06:21:41.400493] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:846) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] failed to close result(close_ret=-4012, ret=-4012) [2024-09-11 06:21:41.400503] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:876) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] failed to process record(executor={ObIExecutor:, sql:"SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE"}, record_ret=-4012, ret=-4012) [2024-09-11 06:21:41.400509] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:902) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] failed to process final(executor={ObIExecutor:, sql:"SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE"}, aret=-4012, ret=-4012) [2024-09-11 06:21:41.400513] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:1691) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] execute sql failed(ret=-4012, tenant_id=1, sql=SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE) [2024-09-11 06:21:41.400519] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:960) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] retry_while_no_tenant_resource failed(ret=-4012, tenant_id=1) [2024-09-11 06:21:41.400526] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1631) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] execute_read failed(ret=-4012, cluster_id=1723810551, tenant_id=1) [2024-09-11 06:21:41.400530] WDIAG [COMMON.MYSQLP] read (ob_single_connection_proxy.cpp:89) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] execute query failed(ret=-4012, sql=SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE, conn=0x7f2f5f0b2050) [2024-09-11 06:21:41.400537] WDIAG [SHARE] fetch_max_id (ob_max_id_fetcher.cpp:479) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] execute sql failed(sql=SELECT VALUE FROM __all_sys_stat WHERE ZONE = '' AND NAME = 'ob_max_used_server_id' AND TENANT_ID = 0 FOR UPDATE, ret=-4012) [2024-09-11 06:21:41.400550] WDIAG [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:270) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] failed to get max id(ret=-4012, tenant_id=1, max_id_type=4, fetch_max_id_type=4) [2024-09-11 06:21:41.400556] INFO [SHARE] fetch_new_max_id (ob_max_id_fetcher.cpp:274) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6] fetch_new_max_id(ret=-4012, ret="OB_TIMEOUT", size=1, tenant_id=1, fetch_id=18446744073709551615, max_id_type=4, fetch_max_id_type=4, id=18446744073709551615, initial=18446744073709551615) [2024-09-11 06:21:41.400631] WDIAG [RS] fetch_new_server_id_ (ob_server_zone_op_service.cpp:738) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=1][errcode=-4012] fetch_new_max_id failed(ret=-4012, ret="OB_TIMEOUT") [2024-09-11 06:21:41.400639] WDIAG [RS] add_servers (ob_server_zone_op_service.cpp:136) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] fail to fetch new server id(ret=-4012, ret="OB_TIMEOUT") [2024-09-11 06:21:41.400643] WDIAG [RS] add_server (ob_root_service.cpp:7917) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] fail to add servers(ret=-4012, ret="OB_TIMEOUT", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.401218] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:1985) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] table scan timeout(ret=-4012, current_time=1726035701400958, table_scan_param_timeout=1726035701348729, lbt()="0x17a30451 0x127e698a 0x5c8a9da 0x5c890b6 0x5c87bc0 0x5c869b0 0x5c8577e 0x5c8365f 0x5e5d615 0x5e5b3b2 0x5e3b377 0x609b40a 0x5fb4ce5 0x5fb391b 0x5fadb02 0x5faae03 0x5faa919 0x5faa612 0x5ebf82e 0x5ebf690 0x1462c07f 0x1462ba6b 0x8ec55bb 0xa1684dd 0xbae8187 0x6153c22 0x5c26066 0x5c1f72a 0xb3c3824 0x17b1814e 0x7f38570081cf 0x7f3856c39d83") [2024-09-11 06:21:41.401229] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:1999) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=11][errcode=-4012] failed to do table scan(ret=-4012, param={tablet_id:{id:117}, ls_id:{id:1}, column_ids:[18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], index_id:117, key_ranges:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:117,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:16, schema_version:1725879596554448, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac817310, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2024-09-11 06:21:41.401424] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:653) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=195][errcode=-4012] failed to do table scan(ret=-4012, &iter=0x7f380d8a71d0, param={tablet_id:{id:117}, ls_id:{id:1}, column_ids:[18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], index_id:117, key_ranges:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:117,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:16, schema_version:1725879596554448, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac817310, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}) [2024-09-11 06:21:41.401589] WDIAG [STORAGE] table_scan (ob_access_service.cpp:334) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=164][errcode=-4012] Fail to scan table, (ret=-4012, ls_id={id:1}, param={tablet_id:{id:117}, ls_id:{id:1}, column_ids:[18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], index_id:117, key_ranges:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:117,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:16, schema_version:1725879596554448, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac817310, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}) [2024-09-11 06:21:41.401747] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_op.cpp:449) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=158][errcode=-4012] fail to scan table(scan_param_={tablet_id:{id:117}, ls_id:{id:1}, column_ids:[18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], index_id:117, key_ranges:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:117,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:16, schema_version:1725879596554448, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac817310, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, ret=-4012) [2024-09-11 06:21:41.401948] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:336) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=200][errcode=-4012] fail to scan table(scan_param_={tablet_id:{id:117}, ls_id:{id:1}, column_ids:[18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], index_id:117, key_ranges:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:117,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1726035701348729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}, sql_mode:4194304, reserved_cell_count:16, schema_version:1725879596554448, query_begin_schema_version:1726022380899392, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f16ac817310, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, fb_read_tx_uncommitted:false, external_file_format:}, external_file_location:""}, ret=-4012) [2024-09-11 06:21:41.402111] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:314) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=163][errcode=-4012] open das task op failed(ret=-4012) [2024-09-11 06:21:41.402115] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:98) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] start das task failed(ret=-4012, *task_wrapper.at(i)={parent:{tenant_id:1, task_id:175055659, op_type:1, errcode:-4012, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f2f01c79930, tx_id:{txid:0}, state:1, addr:"10.21.8.26:2882", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1726035701400958, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180066048, seq_base:1726035701403579, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, snapshot:{this:0x7f37e7dcc8c8, valid:true, source:2, core:{version:{val:1726035701356835000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"10.21.8.26:2882", parts:[], committed:false}, tablet_id:{id:117}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:117}, ls_id:{id:1}, server:"10.21.8.26:2882", in_retry:0, partition_id:117, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:117, access_column_ids:[cnt:16, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1725879596554448, table_param:{table_id:117, output_projector:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:16, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:16, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:17, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:16, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], data:0x7f28efbd8b80, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:16, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], data:0x7f28efbd8bd0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}}, cols_desc:{ObIArray:[cnt:16, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f28efbd8ab0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:16, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f28efbd8cb0, count:16, allocator:0x7f37ccfd06b0, capacity:16, init_cnt:16}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:46, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:64, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:256, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:""}, pd_expr_spec:{calc_exprs:[cnt:1, {type:"T_FUN_SYS_TIME_TO_USEC", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:0x55ff08f215f0, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:1, parent_cnt:0, frame_idx:0, datum_off:1296, res_buf_off:1903, dyn_buf_header_offset:1887, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1312, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6de0}], access_exprs:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:0, pushdown_filters:[cnt:0], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:16, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:162, res_buf_off:1447, dyn_buf_header_offset:1431, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:178, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6210}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:194, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:0, datum_off:243, res_buf_off:1455, dyn_buf_header_offset:1439, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:259, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd62e8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:0, res_buf_off:1393, dyn_buf_header_offset:1377, res_buf_len:46, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:16, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:81, res_buf_off:1439, dyn_buf_header_offset:1423, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:97, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6138}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:324, res_buf_off:1463, dyn_buf_header_offset:1447, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:340, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd63c0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:405, res_buf_off:1487, dyn_buf_header_offset:1471, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:421, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6498}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:486, res_buf_off:1615, dyn_buf_header_offset:1599, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:502, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6570}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:567, res_buf_off:1623, dyn_buf_header_offset:1607, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:583, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6648}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:648, res_buf_off:1647, dyn_buf_header_offset:1631, res_buf_len:64, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:664, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6720}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:729, res_buf_off:1711, dyn_buf_header_offset:1695, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:745, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd67f8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:810, res_buf_off:1735, dyn_buf_header_offset:1719, res_buf_len:128, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:826, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd68d0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:891, res_buf_off:1863, dyn_buf_header_offset:1847, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:907, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd69a8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:972, res_buf_off:1871, dyn_buf_header_offset:1855, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:988, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6a80}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1053, res_buf_off:1879, dyn_buf_header_offset:1863, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1069, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6b58}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1134, res_buf_off:1887, dyn_buf_header_offset:1871, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1150, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6c30}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:66, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:0, datum_off:1215, res_buf_off:1895, dyn_buf_header_offset:1879, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, vector_header_off:1231, eval_vector_func:null, local_session_var_id:-1, this:0x7f28efbd6d08}], is_get:false, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:1726022380899392, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1726035701348729, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}}, scan_range:[{"range":"table_id:117,group_idx:0,(MIN,MIN;MAX,MAX)"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, reserved:0}}) [2024-09-11 06:21:41.402384] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:268) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=268][errcode=-4012] failed to execute aggregated das task(ret=-4012, ret="OB_TIMEOUT", aggregated_task={server:"10.21.8.26:2882", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true) [2024-09-11 06:21:41.402390] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:251) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] failed to execute all das task(ret=-4012) [2024-09-11 06:21:41.402393] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2386) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] execute all das scan task failed(ret=-4012) [2024-09-11 06:21:41.402396] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1604) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] fail to do table scan(ret=-4012) [2024-09-11 06:21:41.402398] WDIAG [SQL.ENG] inner_get_next_row_for_tsc (ob_table_scan_op.cpp:2163) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] failed to init before get row(ret=-4012) [2024-09-11 06:21:41.402400] WDIAG [SQL.ENG] inner_get_next_row_implement (ob_table_scan_op.cpp:2151) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] failed to get next row(ret=-4012) [2024-09-11 06:21:41.402403] WDIAG [SQL.ENG] inner_get_next_row (ob_table_scan_op.cpp:3125) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] get next row failed(ret=-4012) [2024-09-11 06:21:41.402406] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1171) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] inner get next row failed(ret=-4012, type=3, op="PHY_TABLE_SCAN", op_id=0) [2024-09-11 06:21:41.402411] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:147) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] get next row from operator failed(ret=-4012) [2024-09-11 06:21:41.402420] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:466) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=9][errcode=-4012] get next row from exec result failed(ret=-4012) [2024-09-11 06:21:41.402423] WDIAG [SERVER] open (ob_inner_sql_result.cpp:165) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] get_next_row failed(ret=-4012, has_tenant_resource()=true) [2024-09-11 06:21:41.402428] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:706) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] result set open failed(ret=-4012, executor={ObIExecutor:, sql:"SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server"}) [2024-09-11 06:21:41.402432] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:841) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4][errcode=-4012] execute failed(ret=-4012, tenant_id=1, executor={ObIExecutor:, sql:"SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server"}, retry_cnt=0, local_sys_schema_version=1726022380899392, local_tenant_schema_version=1726022380899392) [2024-09-11 06:21:41.402439] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:986) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4012, err_:"OB_TIMEOUT", retry_type:0, client_ret:-4012}, need_retry=false) [2024-09-11 06:21:41.402451] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1251) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f2f01c79930, tx_id:{txid:0}, state:1, addr:"10.21.8.26:2882", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1726035701400958, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180066048, seq_base:1726035701403579, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7f3391a241f8, id:1, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f2f01c79930}, exec_ctx.get_errcode()=-4012) [2024-09-11 06:21:41.402479] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=25][errcode=-4012] result set close failed(ret=-4012) [2024-09-11 06:21:41.402484] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] result set close failed(ret=-4012) [2024-09-11 06:21:41.402487] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:846) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] failed to close result(close_ret=-4012, ret=-4012) [2024-09-11 06:21:41.402493] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:876) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] failed to process record(executor={ObIExecutor:, sql:"SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server"}, record_ret=-4012, ret=-4012) [2024-09-11 06:21:41.402499] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:902) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] failed to process final(executor={ObIExecutor:, sql:"SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server"}, aret=-4012, ret=-4012) [2024-09-11 06:21:41.402502] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:1691) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] execute sql failed(ret=-4012, tenant_id=1, sql=SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server) [2024-09-11 06:21:41.402509] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:960) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] retry_while_no_tenant_resource failed(ret=-4012, tenant_id=1) [2024-09-11 06:21:41.402512] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1631) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] execute_read failed(ret=-4012, cluster_id=1723810551, tenant_id=1) [2024-09-11 06:21:41.402515] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:132) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] query failed(ret=-4012, conn=0x7f3391a24050, start=1726035701401151, sql=SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server) [2024-09-11 06:21:41.402522] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:67) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] read failed(ret=-4012) [2024-09-11 06:21:41.402525] WDIAG [RS] get (ob_server_table_operator.cpp:275) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=2][errcode=-4012] execute sql failed(sql=SELECT *, time_to_usec(gmt_modified) AS last_hb_time FROM __all_server, ret=-4012, ret="OB_TIMEOUT") [2024-09-11 06:21:41.402553] WDIAG [RS] get (ob_server_table_operator.cpp:251) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=5][errcode=-4012] fail to get(ret=-4012, ret="OB_TIMEOUT", proxy_=0x55ff1556f320) [2024-09-11 06:21:41.402560] WDIAG [RS] load_server_manager (ob_server_manager.cpp:1407) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=6][errcode=-4012] get server statuses from all server failed(ret=-4012) [2024-09-11 06:21:41.402564] WDIAG [RS] load_server_manager (ob_root_service.cpp:5950) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3][errcode=-4012] fail to load server manager(ret=-4012) [2024-09-11 06:21:41.402578] WDIAG [RS] add_server (ob_root_service.cpp:7923) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=13][errcode=-4012] fail to load server_manager, please try 'ALTER SYSTEM RELOAD SERVER;'(ret=-4012, ret="OB_TIMEOUT", tmp_ret=-4012, tmp_ret="OB_TIMEOUT") [2024-09-11 06:21:41.402581] INFO [RS] add_server (ob_root_service.cpp:7928) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=3] add server(ret=-4012, ret="OB_TIMEOUT", arg={servers:["10.21.8.23:2882"], zone:"zone1", force_stop:false, op:0}) [2024-09-11 06:21:41.402589] WDIAG [RS] process_ (ob_rs_rpc_processor.h:212) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=7][errcode=-4012] process failed(ret=-4012) [2024-09-11 06:21:41.402593] INFO [RS] process_ (ob_rs_rpc_processor.h:232) [55217][T1_L0_G0][T1][YB420A15081A-000621C87DEFEAEE-0-0] [lt=4] [DDL] execute ddl like stmt(ret=-4012, cost=3651, ddl_arg=NULL)